As how artificial intelligence is revolutionizing endoscopy

Article information

Clin Endosc. 2024;57(3):302-308
Publication date (electronic) : 2024 March 8
doi :
Institut Arnaut Tzanck Gastrointestinal Unt, Saint Laurent du Var, France
Correspondence: Jean-Francois Rey Institut Arnault Tzanck, Saint Laurent du Var 06700, 116 rue du commandant Cahuzac, France E-mail:
Received 2023 September 13; Revised 2023 October 11; Accepted 2023 October 15.


With incessant advances in information technology and its implications in all domains of our lives, artificial intelligence (AI) has emerged as a requirement for improved machine performance. This brings forth the query of how this can benefit endoscopists and improve both diagnostic and therapeutic endoscopy in each part of the gastrointestinal tract. Additionally, it also raises the question of the recent benefits and clinical usefulness of this new technology in daily endoscopic practice. There are two main categories of AI systems: computer-assisted detection (CADe) for lesion detection and computer-assisted diagnosis (CADx) for optical biopsy and lesion characterization. Quality assurance is the next step in the complete monitoring of high-quality colonoscopies. In all cases, computer-aided endoscopy is used, as the overall results rely on the physician. Video capsule endoscopy is a unique example in which a computer operates a device, stores multiple images, and performs an accurate diagnosis. While there are many expectations, we need to standardize and assess various software packages. It is important for healthcare providers to support this new development and make its use an obligation in daily clinical practice. In summary, AI represents a breakthrough in digestive endoscopy. Screening for gastric and colonic cancer detection should be improved, particularly outside expert centers. Prospective and multicenter trials are mandatory before introducing new software into clinical practice.


With constant advances in information technology and its impact in various aspects of our lives, artificial intelligence (AI) algorithms have emerged to enhance machine performance. Unlike machines, the performance of the human brain could be affected by fatigue, stress, or limited experience. AI technology can compensate for human-limited capabilities, prevent human errors, provide machines with reliable autonomy, and increase the productivity and efficiency of work.

The application of AI technology in gastrointestinal (GI) endoscopy has several advantages. It can reduce the inter-operator variability, improve diagnosis accuracy, and facilitate prompt and precise therapeutic decisions on the spot. Furthermore, AI can reduce the time, cost, and the workload associated with endoscopic procedures. However, the implementation of certain guidelines is required.1 Different types of AI computer systems exist to fulfil several functions. The two primary categories of AI systems are computer-assisted detection (CADe) for lesion detection and computer-assisted diagnosis (CADx) for optical biopsy and lesion characterization. Other AI systems offer therapeutic assistance, such as lesion delineation for complete endoscopic resection. Computer-aided quality assurance (CAQ) is the ultimate option. This review focuses on the most advanced AI software available in daily clinical practice, and how it will transform our approach (Table 1).

AI system and related functions


Hassan and Haque2 aimed to draw inferences (bleeding or non-bleeding) using the CADe system by analyzing the spatial domain of an image and extracting features in the frequency domain using complex deep learning (DL). Their goal was to achieve sensitivities and specificities as high as 99% for the detection of GI bleeding. Xiao and Meng3 developed another CADe system to achieve a 99% F1 score or performance score for GI bleeding detection in wireless capsule endoscopy (WCE). The F1 score was calculated using precision and recall scores. They used DL to build CADe with a dataset consisting of 10,000 WCE images, including 2,850 GI bleeding frames and 7,150 normal frames.

AI systems in video capsule endoscopy were among the first to be used in GI endoscopy, and were mainly used for bleeding detection as CADe. However, the most impressive software was published by Ding et al.4 with a detailed diagnosis of all small-bowel abnormalities with a specificity and sensitivity well above that of capsule experts. This software had a reading time of 5.9 minutes against 96.2 minutes from the capsule experts. Even more impressively, Zhang et al.5 developed a gastric magnetically guided capsule with robot for complete gastric diagnosis. This system opens a new era where gastric examination followed by small bowel examination using the same device, with AI for reading.6 This latest development is turning capsule endoscopy into a full AI diagnosis tool.


AI for polyp detection and characterization represents the most advanced tool in computer-aided endoscopy, and some of these techniques are already used in daily practice: (1) CADe for polyp detection and identification7; (2) CADx for polyp characterization and classification (also called optical biopsy or histology prediction)8; (3) CADx for the optical characterization of neoplasia in patients with ulcerative colitis (UC)9; and (4) AI systems can help determine whether additional surgery is needed after endoscopic resection of T1 colorectal cancer by predicting lymph node metastasis.10

Since 2022, Mori et al.11 have developed multiple algorithms designed on an extensive collection of routine colonoscopy videos featuring high-resolution imaging, magnification, and even endocystocopy. Minute and advanced lesions remain a big challenge, even when we have a chance to undergo high-quality colonoscopy screening. In the realm of AI-medicine, AI for colonoscopy is at the forefront, particularly in the number of randomized controlled trials that have showed its effectiveness in detecting a greater quantity of adenomas.11 CADe devices have been tested in colonoscopies, increasing the adenoma detection rate (ADR), mainly in Asian populations.12 Wallace et al.13 reported a significant improvement in the European population, particularly in the detection of diminutive or flat adenomas. This is especially important since many endoscopists are less experienced in identifying these small lesions compared to their Japanese counterparts. In another study, patients presenting for colorectal cancer screening or surveillance were enrolled across four academic centers in the United States of America (USA). They were randomly assigned to either CADe colonoscopy first or high-definition white-light (HDWL) colonoscopy first, followed immediately by the other procedure, all performed by the same endoscopist in a tandem fashion.14 The study cohort included 223 patients and demonstrated a decrease in adenoma and sessile serrated lesion miss rates. There was also an increase in first-pass adenomas per colonoscopy with the use of a CADe system when compared with HDWL colonoscopy alone. Additionally, benefits were observed for proximal colon lesions15 and for missing polyps in a large meta-analysis.16

Characterization is another major expected benefit, as routine classifications, such as narrow-band imaging (NBI), International Colorectal Endoscopic, or Japan NBI Expert Team classifications, are not properly applied. Hossain et al.17 showed a marked improvement in characterization using AI: CADx with up to 90.9% sensitivity (vs. 48.1% without AI), 95.8% negative predictive value, 80% accuracy with white-light endoscopy, and 84.6% accuracy using image-enhanced endoscopy.

Aside from detection and characterization, other benefits are expected, such as polyp size and histology of colorectal polyps, which are essential factors in appropriate management. However, a query arises regarding how accurately we typically diagnose the polyp size. Most endoscopists would probably not answer this question because they have few opportunities to confirm the accuracy of their own estimations. Kwak et al.18 demonstrated the usefulness of a newly developed computer-aided tool based on AI for the accurate measurement of colorectal polyps, even with the pitfalls linked to endoscope handling.

Liu et al.14 developed an AI system to measure the fold-examination quality (FEQ) of colonoscopy. The system’s evaluation of FEQ was strongly correlated with FEQ scores from experts, historical ADR, and withdrawal time of each colonoscopist. Other softwares proposed full monitoring: colonic preparation, time to reach the cecum, warning for blind spots, and recording onsite endoscopic report, which is the ultimate goal.19

In summary, CADe should be used worldwide for screening colonoscopies based on the World Endoscopy Organization position statement20: (1) CADe for colorectal polyps is likely to improve the effectiveness of colonoscopy by reducing the adenoma miss rates, thereby increasing adenoma detection. (2) In the short-term, the use of CADe is likely to increase the healthcare costs by detecting more adenomas. (3) In the long-term, the increased cost of CADe could be balanced by savings in costs related to cancer treatment (surgery, chemotherapy, or palliative care) owing to CADe-related cancer prevention. (4) Healthcare delivery systems and authorities should evaluate the cost-effectiveness of CADe in order to support its use in clinical practice. (5) CADx for diminutive polyps (≤5 mm), when it has sufficient accuracy, is expected to reduce the healthcare costs by reducing the number of unnecessary polypectomies, pathological examinations, or both. As a significant number of hyperplastic polyps are removed, the discard policy allows the resection of benign adenomas without further costly pathological examinations.21


Early detection is essential for improving the prognosis and mortality of gastric cancer, particularly in countries with high incidence. Early detection also allows for minimally invasive endoscopic resection, which has been shown to have excellent overall survival comparable to gastrectomy, while preserving stomach function. Without AI, the false-negative rates of gastric cancer during screening endoscopy range from 4.6% to 25.8%. Multiple studies have demonstrated the benefits of AI for detection and characterization (assessing the depth of invasion) based on Japanese classifications, detecting Helicobacter pylori infection, and making endoscopic resection more accurately with delineation of the cancer area.22

Ishioka et al.23 developed an artificial diagnostic support tool, ‘Tango’, to differentiate early gastric cancers (EGCs) using still images of 150 neoplastic and 165 non-neoplastic lesions. Tango achieved superiority over the specialist based on sensitivity (84.7% vs. 65.8%; difference, 18.9%; 95% confidence interval [CI], 12.3%–25.3%) and demonstrated non-inferiority based on accuracy (70.8% vs. 67.4%). More importantly, in clinical practice, Tango achieved superiority over the non-specialist based on sensitivity (84.7% vs. 51.0%) and accuracy (70.8% vs. 58.4%). With the same goal, Wu et al.24 reported a prospective study that included 1,050 patients using ENDOANGEL software (Jinshan) for the detection of EGC and monitoring the quality of gastric examination. The ENDOANGEL group had fewer blind spots (5.38 vs. 9.32, p<0.001) and detected 196 lesions and three EGCs with a per-lesion accuracy of 84.7%, sensitivity of 100%, and specificity of 84.4% for detecting gastric cancer. Similarly, intestinal metaplasia detection is improved by AI.25 CADx for gastroscopy is expected to serve as a second observer during real-time gastroscopy, helping endoscopists detect more neoplasms. Simultaneously, it serves as an educational tool for training novice endoscopists.


Esophageal cancer (EC) is a common and severe disease worldwide. Although the morbidity associated with esophageal squamous cell carcinoma has decreased in recent years, it remains the predominant histological type of EC in some regions (Asia and Africa). Most patients are diagnosed at advanced stages of the disease. Therefore, early detection is crucial to improve treatment and survival. High-resolution endoscopy with NBI and magnification is the standard for detection and characterization.

Everson et al.26 demonstrated the benefit of AI in a panel of both European and Asian endoscopists compared with the goal standard, even with different pathological assessments in different areas, but based on intrapapillary capillary loop endoscopic classification. Expert European and Asian endoscopists attained F1 scores (a measure of binary classification accuracy) of 97.0% and 98%, respectively. The sensitivity and accuracy of the European and Asian clinicians were 97%, 98%, 96.9%, and 97.1%, respectively. However, even more importantly, Yuan et al.27 reported the benefit of AI in obtaining better delineation. Non-magnified NBI images (10,047 still pictures and 140 videos from 1,120 patients and 1,183 lesions) from four hospitals were collected and annotated. The delineation performance of the system was compared with that of the endoscopists. Furthermore, the system was directly integrated into the endoscopy equipment, and its real-time diagnostic capability was prospectively estimated. The accuracy was 91.4% for detecting lesions and 85.9% for delineating lesions. As expected, the benefit was superior to that of junior doctors and similar to that of experts.

In Western countries, esophageal adenocarcinoma has been linked with Barrett's esophagus (BE). The prognosis is strongly related to the stage of diagnosis. However, >40% of patients are diagnosed after the disease has metastasized, with a survival rate of <20%. Adenocarcinomas in BE patients are often preceded by high-grade dysplasia. Patients with BE undergo regular surveillance to detect neoplasia at an early stage, and can be treated with endoscopic resection. Detailed characterization requires expertise or AI.28 Abdelrahim et al.29 carried out a multicenter study for the detection and localization of Barrett’s neoplasia and assessed its performance compared with that of general endoscopists by using real-time video sequences. The CADx system detected Barrett’s neoplasia with a sensitivity, specificity, negative predictive value, and accuracy of 93.8%, 90.7%, 95.1%, and 92.0%, respectively, compared with the endoscopists’ performance of 63.5%, 77.9%, 74.2%, and 71.8%, respectively (p<0.05 in all parameters). The CADx system localized neoplastic lesions with an accuracy, mean precision, and mean intersection over union of 100%, 0.62, and 0.54, respectively. This promising result for the common situation in Europe and USA should be tested in prospective clinical trials.


Inflammatory bowel diseases, such as UC and Crohn’s disease, are chronic conditions for which endoscopic diagnosis and assessment require considerable clinical expertise. A future benefit will be the introduction of AI in clinical trials in a group of patients. This will help prevent clinical bias and eliminate interobserver variation. In summary, I agree with Murino and Rimondi,30 “AI is going to drastically change our approach to diagnosis endoscopy. In contrast to its human counterpart, AI can manage an exceptional amount of data simultaneously, does not get fatigued, and can be highly effective and efficient”, especially in patients with small-bowel disease. At present, with respect to endoscopic retrograde cholangiopancreatography and endoscopic ultrasonography, AI softwares are at an early stage.31-33 In the future, AI software will be developed for quality assurance, monitoring colonoscopy, and producing immediate onsite endoscopic reports. This represents a significant potential improvement in daily practice, particularly for non-expert units.34


As medical knowledge progresses, the use of generative AI in practice depends on its ability to provide up-to-date information. Although these tools can access the most recent data, their ability to convey and account for data that change over time remains unclear. Similarly, there is a need to clarify data originating from various sources, including differentiating between established clinical standards and emerging research. This complexity is compounded in situations where multiple current resources may not be in agreement (e.g., different guidelines on colorectal cancer screening). To accept AI algorithms in clinical practice, their effectiveness, clinical value, and reliability must be rigorously assessed. Parasa et al.1 provided a guiding framework for all stakeholders in endoscopy. At present, there is no AI ecosystem regarding the standards, metrics, and evaluation methods for emerging and existing AI applications to aid in their clinical adoption and implementation. They also provide guidance and best practices for the evolution of AI technologies as they mature in the endoscopy space. The model cart for AI in endoscopy includes multiple groups of features: basic information about the model, product details, clinical implications, performance evaluation, explainable/trustworthy AI, model development facts, and postmarked/real-world data aside from all factors. It is critical in clinical practice to avoid a multitude of false alert-disturbing endoscopic examinations. This American Society for Gastrointestinal Endoscopy (ASGE) technical document is important for all AI developers. If CADe is effective, it remains to be understood why its adoption in clinical practice appears to be progressing slowly. The drawbacks are mainly related to costs (without healthcare system reimbursement), and products are commercially available without proper evaluation. Excessive false positive results disrupt the endoscopist’s attention and can affect the overall results. For colorectal cancer screening, an increase in ADR has proved to decrease the post-colonoscopy cancer risk, but not yet with an AI-associated gain.8 The resect-and-discard strategy based on characterization and expected histology is still a subject of debate in several countries outside USA and could confuse many endoscopists.35 Developers should follow the recent guidelines from ASGE in all cases.1


At the fundamental level, these models are trained on a breadth of data beyond those accessible to most individuals and require accurate databases.36 The synthesis of this information presents an opportunity to broaden access and may aid in reducing disparities in underserved communities. However, this information alone provides little utility if it cannot be understood. Health literacy remains a barrier in providing usable responses to complex health-related questions. The response readability of ChatGPT exceeded the 8th-grade level, limiting its utility for a subset of patients and potentially widening the gap in healthcare access.37


I believe that the future of GI endoscopy will undergo dramatic transformation in the upcoming years with the integration of AI into this field (Table 2). Manufacturers have made huge investments in this promising technology, and the results have started to emerge. Currently, endoscopists are responsible for performing, detecting, analyzing, and providing treatment independently. In addition, they must possess extensive experience and enormous knowledge to offer the best care to their patients. They have to maintain good memory and practical skills over time while staying updated on every new piece of information, recommendation, and guideline.

Steps for how AI will revolutionize endoscopy


Conflicts of Interest

The author has no potential conflicts of interest.




1. Parasa S, Repici A, Berzin T, et al. Framework and metrics for the clinical use and implementation of artificial intelligence algorithms into endoscopy practice: recommendations from the American Society for Gastrointestinal Endoscopy Artificial Intelligence Task Force. Gastrointest Endosc 2023;97:815–824.
2. Hassan AR, Haque MA. Computer-aided gastrointestinal hemorrhage detection in wireless capsule endoscopy videos. Comput Methods Programs Biomed 2015;122:341–353.
3. Xiao Jia, Meng MQ. A deep convolutional neural network for bleeding detection in wireless capsule endoscopy images. Annu Int Conf IEEE Eng Med Biol Soc 2016;2016:639–642.
4. Ding Z, Shi H, Zhang H, et al. Gastroenterologist-level identification of small-bowel diseases and normal variants by capsule endoscopy using a deep-learning model. Gastroenterology 2019;157:1044–1054.
5. Zhang Y, Zhang Y, Huang X. Development and application of magnetically controlled capsule endoscopy in detecting gastric lesions. Gastroenterol Res Pract 2021;2021:2716559.
6. Rey JF. Gastric examination by guided capsule endoscopy: a new era. Lancet Gastroenterol Hepatol 2021;6:879–880.
7. Gimeno-García AZ, Hernández Negrin D, Hernández A, et al. Usefulness of a novel computer-aided detection system for colorectal neoplasia: a randomized controlled trial. Gastrointest Endosc 2023;97:528–536.
8. Rex DK, Berzin TM, Mori Y. Artificial intelligence improves detection at colonoscopy: why aren’t we all already using it? Gastroenterology 2022;163:35–37.
9. Maeda Y, Kudo SE, Ogata N, et al. Use of advanced endoscopic technology for optical characterization of neoplasia in patients with ulcerative colitis: systematic review. Dig Endosc 2022;34:1297–1310.
10. Ichimasa K, Kudo SE, Lee JW, et al. Artificial intelligence-assisted treatment strategy for T1 colorectal cancer after endoscopic resection. Gastrointest Endosc 2023;97:1148–1152.
11. Mori Y, Misawa M, Kudo SE. Challenges in artificial intelligence for polyp detection. Dig Endosc 2022;34:870–871.
12. Glissen Brown JR, Mansour NM, Wang P, et al. Deep learning computer-aided polyp detection reduces adenoma miss rate: a United States multi-center randomized tandem colonoscopy study (CADeT-CS Trial). Clin Gastroenterol Hepatol 2022;20:1499–1507.
13. Wallace MB, Sharma P, Bhandari P, et al. Impact of artificial intelligence on miss rate of colorectal neoplasia. Gastroenterology 2022;163:295–304.
14. Liu W, Wu Y, Yuan X, et al. Artificial intelligence-based assessments of colonoscopic withdrawal technique: a new method for measuring and enhancing the quality of fold examination. Endoscopy 2022;54:972–979.
15. Shaukat A, Lichtenstein DR, Somers SC, et al. Computer-aided detection improves adenomas per colonoscopy for screening and surveillance colonoscopy: a randomized trial. Gastroenterology 2022;163:732–741.
16. Sivananthan A, Nazarian S, Ayaru L, et al. Does computer-aided diagnostic endoscopy improve the detection of commonly missed polyps?: a meta-analysis. Clin Endosc 2022;55:355–364.
17. Hossain E, Abdelrahim M, Tanasescu A, et al. Performance of a novel computer-aided diagnosis system in the characterization of colorectal polyps, and its role in meeting Preservation and Incorporation of Valuable Endoscopic Innovations standards set by the American Society of Gastrointestinal Endoscopy. DEN Open 2022;3e178.
18. Kwak MS, Cha JM, Jeon JW, et al. Artificial intelligence-based measurement outperforms current methods for colorectal polyp size measurement. Dig Endosc 2022;34:1188–1195.
19. Yao L, Zhang L, Liu J, et al. Effect of an artificial intelligence-based quality improvement system on efficacy of a computer-aided detection system in colonoscopy: a four-group parallel study. Endoscopy 2022;54:757–768.
20. Mori Y, East JE, Hassan C, et al. Benefits and challenges in implementation of artificial intelligence in colonoscopy: World Endoscopy Organization position statement. Dig Endosc 2023;35:422–429.
21. Rondonotti E, Hassan C, Tamanini G, et al. Artificial intelligence-assisted optical diagnosis for the resect-and-discard strategy in clinical practice: the Artificial intelligence BLI Characterization (ABC) study. Endoscopy 2023;55:14–22.
22. Abe S, Oda I. How can endoscopists adapt and collaborate with artificial intelligence for early gastric cancer detection? Dig Endosc 2021;33:98–99.
23. Ishioka M, Osawa H, Hirasawa T, et al. Performance of an artificial intelligence-based diagnostic support tool for early gastric cancers: retrospective study. Dig Endosc 2023;35:483–491.
24. Wu L, He X, Liu M, et al. Evaluation of the effects of an artificial intelligence system on endoscopy quality and preliminary testing of its performance in detecting early gastric cancer: a randomized controlled trial. Endoscopy 2021;53:1199–1207.
25. Siripoppohn V, Pittayanon R, Tiankanon K, et al. Real-time semantic segmentation of gastric intestinal metaplasia using a deep learning approach. Clin Endosc 2022;55:390–400.
26. Everson MA, Garcia-Peraza-Herrera L, Wang HP, et al. A clinically interpretable convolutional neural network for the real-time prediction of early squamous cell cancer of the esophagus: comparing diagnostic performance with a panel of expert European and Asian endoscopists. Gastrointest Endosc 2021;94:273–281.
27. Yuan XL, Zeng XH, Liu W, et al. Artificial intelligence for detecting and delineating the extent of superficial esophageal squamous cell carcinoma and precancerous lesions under narrow-band imaging (with video). Gastrointest Endosc 2023;97:664–672.
28. Meinikheim M, Messmann H, Ebigbo A. Role of artificial intelligence in diagnosing Barrett’s esophagus-related neoplasia. Clin Endosc 2023;56:14–22.
29. Abdelrahim M, Saiko M, Maeda N, et al. Development and validation of artificial neural networks model for detection of Barrett’s neoplasia: a multicenter pragmatic nonrandomized trial (with video). Gastrointest Endosc 2023;97:422–434.
30. Murino A, Rimondi A. Automated artificial intelligence scoring systems for the endoscopic assessment of ulcerative colitis: how far are we from clinical application? Gastrointest Endosc 2023;97:347–349.
31. Marya NB, Powers PD, Petersen BT, et al. Identification of patients with malignant biliary strictures using a cholangioscopy-based deep learning artificial intelligence (with video). Gastrointest Endosc 2023;97:268–278.
32. Huang L, Xu Y, Chen J, et al. An artificial intelligence difficulty scoring system for stone removal during ERCP: a prospective validation. Endoscopy 2023;55:4–11.
33. Kuwahara T, Hara K, Mizuno N, et al. Artificial intelligence using deep learning analysis of endoscopic ultrasonography images for the differential diagnosis of pancreatic masses. Endoscopy 2023;55:140–149.
34. Houwen BB, Nass KJ, Vleugels JL, et al. Comprehensive review of publicly available colonoscopic imaging databases for artificial intelligence research: availability, accessibility, and usability. Gastrointest Endosc 2023;97:184–199.
35. Rex DK. Making a resect-and-discard strategy work for diminutive colorectal polyps: let’s get real. Endoscopy 2022;54:364–366.
36. Yang CB, Kim SH, Lim YJ. Preparation of image databases for artificial intelligence algorithm development in gastrointestinal endoscopy. Clin Endosc 2022;55:594–604.
37. Feldman K, Nehme F. Beyond clinical accuracy: considerations for the use of generative artificial intelligence models in gastrointestinal care. Gastroenterology 2023;165:336–338.

Article information Continued

Table 1.

AI system and related functions

AI system category Areas of assistance
Technical Scope guidance for colonoscope insertion
Detection (CADe) Polyp detection
Barrett’s neoplasia detection
Bleeding detection
Helicobacter pylori detection
Diagnostic (CADx) Early cancer identification
Cancer staging (estimation of invasion depth)
Polyp characterization or classification
Diagnosis of normal vs. inflammatory mucosa in inflammatory bowel disease
Gastrointestinal disease prediction from patient data
Therapeutic Lesion delineation
Assistance in therapeutic decisions
Risk stratification, prediction of outcome, and potential need
for therapeutic intervention (in gastrointestinal bleeding)
Capsule endoscopy Full detailed diagnosis
Magnetically guided gastric capsule with robot

AI, artificial intelligence; CADe, computer-assisted detection; CADx, computer-assisted diagnosis.

Table 2.

Steps for how AI will revolutionize endoscopy

Step Development
1 Software conception
 What software can do
 How it will perform
 Quality of endoscopic material (still pictures, recorded videos)
2 During endoscopic procedures
 Reduced false alarm and operator distraction
 Clear explanation of system performance
3 Post-market evaluation
 Accurate analysis of software performance
 Practitioner feedback
4 Immediate and corrected new update version

AI, artificial intelligence.