JIMT

According to cancer registration statistics in Korea in 2019, colorectal cancer (CRC) was newly diagnosed with 29,030 cases, which is the fourth most common cancer with 56.5 cases per 100,000 population¹. It is known that about 70% of CRC develop through the adenoma-carcinoma sequence, and the other 25-30% through the serrated pathway^2,3. Because detection and excision at the polyp stage by colonoscopy can reduce the incidence and mortality of CRC^4,5, many countries are implementing screening programs through evidence-based guidelines⁶.

Current CRC screening procedures in Korea include fecal occult blood test (FOBT) as a screening test, which additionally requires the colonoscopy when the test is positive⁷. This screening procedure has a low participation rate owing to patients’ discomfort and reluctance during the stool collection process^8,9, and it takes up to approximately 15 days to confirm the result. Additionally, when evaluating medical device diagnostic tools, a low sensitivity increases the likelihood of misdiagnosing a diseased patient as normal. When several studies conducted by immunochemical FOBT were combined, the false positive rate for CRC screening was reported to be 2%–12% and the false-negative rate to be 21%–50%⁷; therefore, there is a considerable possibility of missing CRC.

Recently, as part of a strategy to efficiently screen for colorectal polyps and CRC, an artificial intelligence (AI)-based disease prediction software that can complement the problems of FOBT has been developed. When an examinee enters the questionnaire information through a web/mobile application (Fig. 1A), the risk of occurrence of colorectal adenomas and high-risk adenomas is predicted, and the results and recommendations are instantaneously presented. This can help determine whether to perform a colonoscopy, and it is possible to omit the FOBT or double contrast barium enema test, thereby relieving patient discomfort and reluctance to participate in the screening test as well as reducing the time and economic cost. From a health care provider’s point of view, a virtuous cycle structure can be expected, in which the patient treatment process becomes easier by providing improved medical services and auxiliary diagnostic results.

Figure 1. Health-screening questionnaire through web/mobile application. (A) Initial screen. (B) Appearance of test result.

The purpose of this study was to evaluate the applicability of AI-based algorithm to predict the risk of colorectal and high-risk adenomas using a health-screening questionnaire in an actual clinical field setting.

AI-based diagnostic prediction software

The online software (Dr. Answer by PCT Co., Ltd.) used in this study was an AI model that learned and validated 77,888 multi-center health examination lifestyle questionnaire data, colonoscopy results, and biopsy results between August 2008 and December 2019. It analyzes online questionnaire information to predict the risk of developing colorectal adenomas and high-risk adenomas. Predictions of the risk of developing adenomas had a specificity of 83% and a sensitivity of 86%, and those high-risk adenomas had a specificity of 89% and a sensitivity of 81%.

Study design

This was a prospective study conducted at a single institution, and data were acquired using a questionnaire from a developed web/mobile application. The risk of colorectal and high-risk adenomas were predicted, and colonoscopy and biopsy results were collected and analyzed. The inclusion criteria were those aged between 18 and 80 years who agreed to provide health checkup information among asymptomatic colonoscopy subjects. The exclusion criteria were the need for colonoscopy to confirm colorectal disease after diagnosis at an external hospital or if colonoscopy had been performed within the last 5 years. Dropout was performed when the entire colon was not examined by colonoscopy or consent was withdrawn.

In the process of data collection, the subject selection criteria were checked and informed consent was obtained, health-screening questionnaire information was entered through a web/mobile application, and then the prediction results were presented in real time. After colonoscopy, test results were collected, and a satisfaction survey was conducted through a questionnaire. In addition, pathologic results were collected when polyps were excised or biopsies were performed.

The sample size was calculated to be 451 at a 95% confidence level and 4% margin of error based on a 25% adenoma detection rate, which is the quality control index for colonoscopy. Assuming a dropout rate of 10%, the sample size target was 500. This study was approved by the Institutional Review Board of the National Cancer Center, Korea (NCC 2020-0147), and informed consent was obtained from all the patients before data collection.

Online questionnaire information collection

The online health checkup questionnaire consists of 53 questions, and by topic, it is composed of general information, past medical history, family history, gynecological questionnaire (female), lifestyle, symptoms, and dietary information. The details are presented in Table 1.

Table 1 . Online health checkup questionnaire.

Classification	Content
General information	Marital status, Occupation, Last health checkup etc.
Past medical history	Surgery history, medication history, gastrointestinal disease, colon and anus disease, liver disease, etc.
Family history	Family history of any cancer in first-degree relatives
Gynecological interview (female only)	Menstrual status, postmenopausal medication history, etc.
Lifestyle	Smoking, alcohol drink, exercise, etc.
Symptom questionnaire	Physical discomfort symptoms, past fecal occult blood test results, etc.
Dietary information	Number of meals, amount, type of food consumed, etc.

Interpretation of test results

The results are shown in Fig. 1B. The risk of adenoma was classified into four groups according to the result value, and the results and recommendations were presented together.

1) Group A (approximately 10% or less): Interest level. Get regular interest.

2) Group B (more than 10% to less than 30%): Needs attention level. Be mindful of your lifestyle.

3) Group C (> 30% to < 50%): Caution level. Endoscopy is recommended even in the absence of symptoms.

4) Group D (>50%): Warning level. Please have endoscopy.

In the case of high-risk adenomas, the risk of occurrence was low, so only predicted numerical values and general information were provided.

Study outcomes

We defined high-risk adenoma as adenoma larger than 10 mm, three or more, accompanied by tubulovillous/villous histology or high-grade dysplasia¹⁰.

Our main results assessed the accuracy of the prediction of adenoma risk using the AI-based algorithm was evaluated.

Accuracy=(True Positive+True Negative)/(True Positive+ True Negative+False Positive+False Negative)

As a secondary result, the prediction accuracy of high-risk adenomas was evaluated. In addition, related items of diagnostic methods such as sensitivity, specificity, positive predictive value, negative predictive value, receiver operating characteristic (ROC) curve were analyzed. And the results of the satisfaction survey on application use were analyzed.

Data from 516 people were collected from September 21, 2020, to November 12, 2020 (approximately 8 weeks), and data from 509 people, excluding 5 people who had network errors and two people who had input errors during the data entry process, were used for analysis.

The characteristics of the test participants are presented in Table 2. The male-to-female ratio was 50.3% and 49.7%, median age was 59 years, median body mass index (BMI) was 23.6 kg/m², and BMI of 25 kg/m² or more was 35.2%. A family history of CRC in first-degree relatives was 8.6%, smoking history was 39.7%, and alcohol drinking history was 53.2%. Colonoscopy revealed a polyp detection rate of 55.4%, adenoma detection rate of 42.2%, high-risk adenoma detection rate of 6.3%, and cancer in four patients (0.8%).

Table 2 . Patients’ characteristics and results of colonoscopy.

Characteristic	Results (509 patients)
Sex
Female	256 (50.3)
Male	253 (49.7)
Age (yr)	59 (23–80)
BMI (kg/m2)	23.6 (17.0–41.9)
High BMI (25 kg/m2 or more)	179 (35.2)
Family history of colorectal cancer in a first-degree relative	44 (8.6)
Smoking history	202 (39.7)
Alcohol drinking history	271 (53.2)
Results of colonoscopy
Number of people with polyps	282 (55.4)
Number of people with adenomas	215 (42.2)
Number of people with high-risk adenomas*	32 (6.3)
Number of people with cancers	4 (0.8)

Values are presented as number (%) or median (range)..

BMI: body mass index..

*High-risk adenoma includes tubular adenoma ≥10 mm, 3 or more adenomas, adenoma with villous histology, or high-grade dysplasia..

The detection rate of adenoma and high-risk adenoma, divided into those under 40s, 40s, 50s, 60s, and 70s or older according to age, increased with age (Fig. 2A). The detection rates of adenomas (34.4% vs. 50.2%, P<0.001) and high-risk adenomas (3.5% vs. 9.1%, P=0.016) were significantly higher in men (Fig. 2B).

Figure 2. Colonoscopy results of according to age group (A), by sex (B), and by risk level (C) computed by artificial intelligence risk calculator. HRA: high-risk adenoma.

When divided into four groups according to the predictive diagnosis stage classified by the risk (%) of colorectal adenoma analyzed using online software in Fig. 2C: group A (less than 10%), group B (10% to less than 30%), group C (less than 30% to 50%), and group D (more than 50%), the adenoma detection rate was 19.0%, 31.4%, 43.6%, and 73.0%, respectively, and the high-risk adenoma detection rate was 4.8%, 4.8%, 6.6%, and 8.1%, respectively.

The relationship between the distribution of risk scores and biopsy results is shown in Fig. 3. Fig. 3A shows that there was a statistically significant difference in the risk score between the adenoma positive and adenoma-negative groups (40.8±10.9 vs 35.8±13.2; P<0.001), and there was no difference in the risk score for high-risk adenoma in Fig. 3B (15.4±11.2 vs 12.5±11.0; P=0.155).

Figure 3. Differences and distribution of the risk by pathologic results on adenoma (A), high-risk adenoma (B), adenoma under age 50 years (C).

Based on the cutoff of 46% obtained with 1,000 bootstrap samples, the item for the diagnostic method, the primary outcome variable, had an accuracy of 60.3% (95% confidence interval [CI], 56.2%–64.2%). And sensitivity was 48.8% (95% CI, 42.3%–55.4%), specificity 68.7% (95% CI, 63.6%–73.8%), positive predictive value 53.3% (95% CI, 48.1%–58.6%), negative predictive value 64.7% (95% CI, 61.4%–68.2%) (Fig. 4). The ROC curves are shown in Fig. 5, with an area under the curve (AUC) of 0.619 (95% CI, 0.570–0.668) for adenoma risk and 0.583 (95% CI, 0.478–0.688) for high-risk adenoma risk. When the cutoff value is divided by 30%, 46%, and 50%, each item is shown in Table 3.

Table 3 . Sensitivity, specificity, PPV, NPV, and accuracy of adenoma risk prediction for each cutoff value.

Cutoff value (%)	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)	Accuracy (%)
30	82.8	30.6	46.6	70.9	52.7
46*	48.8	68.7	53.3	64.7	60.3
50	12.6	96.6	73.3	60.2	61.1

PPV: positive predictive value, NPV: negative predictive value..

*Cutoff value from 1,000 bootstrap samples..

Figure 4. Cross table and statistical values of adenoma at the cutoff value derived through 1,000 bootstraps. CI: confidence interval.

Figure 5. Receiver operating characteristic (ROC) curve for adenoma and high-risk adenoma. AUC: area under the curve.

The questionnaire is shown in Fig. 6 and has 8 items, consisting of 2 convenience items, three satisfaction items, one aesthetic item, and two reliability items. Each question consisted of a 5-step evaluation answers, and the proportion of positive evaluations ranged from 75.3% to 89.3% for the 8 items.

Figure 6. Satisfaction results of surveys through the application.

The FOBT, a CRC screening test currently being implemented in Korea, has limitations in the test itself, as well as the examinee's discomfort with the actual stool collection process^8,9. Moreover, it has a low screening rate compared with other cancers owing to the delay in reporting the results and requirement for a follow-up colonoscopy when the test is positive^11,12. Therefore, the clinical applicability of the method for determining the need for colonoscopy was evaluated by predicting the risk of colonic neoplasms in a simple and non-invasive way using an AI-based algorithm and presenting the results immediately.

This study prospectively recruited 509 patients using an AI-based application developed through learning and verifying data of 77,888 people from three health institutions, comprising data from questionnaire response and colonoscopy and biopsy results. As a result, the adenoma detection rate was reported to be 42.2%, and the accuracy of finding colonic adenomas, which was the primary outcome, was 60.3%. Upon classifying the risk of adenomas into four stages, the higher the risk, the higher the adenoma positivity, and the survey participants gave a positive answer of 75% or more for using the application.

Regarding the pathogenesis of CRC, it is known that resection through colonoscopy in the polyp stage prevents cancer and reduces mortality, and risk factors for colorectal polyps have been identified in many studies. In particular, smoking, drinking, and high BMIs are highly correlated with increased risk, while there are factors that increase risk, such as red meat and fat. Fiber, calcium, folic acid, vitamin D, aspirin, and physical activity are known to reduce risk^13-16.

Previously published studies predicted the risk of advanced neoplasia or high-risk adenomas according to each study’s definition, presented risk reduction strategies that had undergone internal validation^17-19 and external validation using risk models from previous studies^20,21, and compared head-to-head with 17 risk models from previously published studies. The AUC for the risk score of high-risk adenoma or CRC ranged from 0.58 to 0.65, with modest discriminatory power¹⁶.

Recently, AI has been used to discover and predict patterns in massive data, and in Hussan’s study, machine learning techniques showed significantly improved discrimination compared to logistic regression in predicting early onset CRC²².

Lee et al.²³ reported a retrospective study that had limitations such as lack of information on diet and physical activity, but they developed a non-invasive colorectal polyp risk stratification tool using AI. Sex (male, female), age group (old, young) divided by age 50, and BMI (normal, overweight, obese, underweight) divided into 4 stages were combined and divided into 16 subgroups to show discrimination through AUC values; values ranged from 0.61 to 0.91, and were generally high in the obese combination²³.

The limitations of our study are as follows: First, the AI-based application was developed using data from three institutions, and data were collected from a single institution in this study. So, due to the limitations of the learning data, there are limitations in applying it to other racial or ethnic groups.

Second, although patterns were discovered and risks were scored on medical data sources through machine learning, the respective weight of the factors are not known owing to the black-box nature. And as the criteria for dividing groups presented in Fig. 2C (10%, 30%, 50%) is arbitrary.

Third, when the AI model was developed, the reported specificity and sensitivity of adenoma risk prediction were 83% and 86%, and the high-risk adenoma risk had 89% specificity and 81% sensitivity, but the results of this study showed lower values compared to those. In particular, the value of approximately 46% presented as the cutoff value was the most frequent, as shown in Fig. 2A. It is thought that the discrimination power can be improved by learning more data or adding factors such as CBC²⁴, blood vitamin D concentration^25,26, and blood lipid profile^27,28, which are weak but known risk factors for polyps as it is included in general health examinations.

Fourth, similar trend was confirmed in Fig. 1A and Fig. 1C. From this, it can be assumed that the weight of the age variable was large in the developed tool. Although the prevalence of CRC is increasing in younger age groups who are not currently subjects of national screening programs, the results of this study showed no difference in risk according to the presence or absence of adenoma in those <50 years of age (Fig. 2C). In the future, targeting patients under the age of 50 years who are not currently subjects of national cancer screening is expected to be a good supplementary tool. One study reported discriminative power of AUC 0.57 to 0.64 with 4 types of machine learning in subjects aged 35 to 50 years²².

However, the strength of our study is that it prospectively recruited a patient group and collected feedback to a lifestyle-related questionnaire to evaluate the applicability of an AI-based application to actual clinical processes, and the satisfaction metric of the AI-based application was also evaluated by the examinee.

In applying this software, most of the questionnaire items were general and not specific to CRC; therefore, it can be used for screening other diseases at the same time. In addition, if risk information related to the questionnaire items are also provided, it can also serve as a good educational program. In particular, for high-risk subjects, educational materials about bowel preparation are provided along with video information, which can help increase the adenoma detection rate. Through this utilization, it will be possible to expand this into an easy-to-use cancer prevention platform service.

In addition, because the risk score appears immediately, it can help examinees and doctors rapidly determine the need for colonoscopy. And, recently, many studies on colonoscopy polyp detection using AI have been conducted and used in the field; hence, a synergistic effect can be expected²⁹.

In conclusion, we prospectively surveyed health-screening questionnaires using a familiar web/mobile application format. Through this, the risks of adenoma and high-risk adenomas were calculated using AI and presented as a numerical value, and the applicability to actual clinical practice was evaluated by comparison with colonoscopy results. This method has the potential to complement the inconvenient and cumbersome current screening method, and if it is improved through continuous data learning in the future, it can be used for effective CRC prevention.

No potential conflict of interest relevant to this article was reported.

This work was supported by the National Research Foundation of Korea (NRF) grant (NRF-2022R1A2C2009757) and the grant from National Cancer Center Korea (NCC-2010030).

This study was presented in the [Best investigator – Colorectal cancer] session at The Korean Society of Coloproctology 56th Annual Meeting in Gyeong-ju in March 2023.