In Pursuit of an Expert l Intelligence System: Reproducing Human Physicians Diagnostic Reasoning and Triage Decision Making
- pp28-39
Aprimary objective in the development of an l intelligence (AI) system should be to replicate an expert physician’s thought process. s study compares a hypothetico-deductive powered system (Doctor Ai) and a decision tree powered sys tem (Babylon Health Ai) with human physicians to evaluate , measured by the time needed to (1) diagnose and (2)make triage decisions. In this study, both AI systems and the physicians evaluated a total of n typical textbook presentations of clinical scenarios. e study found that both AI systems agreed on patient disposition decisions for 93% of cases (14out of 15 cases; p<0.08) with no statistically t e from physicians, indicating that both AI systems are equally e in patient triage decisions relative to the physicians. e Doctor Ai system agreed with the physicians on thenal diagnosis for 73.3% (11 out of 15) of the cases, while Babylon Health Ai provided a l diagnosis in only 53% (8 out of15) of cases. For the remaining cases, the diagnosis was either undisclosed or could not be determined. In this study, DoctorAi used an average of 7.8 (±2.08) computer screens to reach diagnostic n compared to Babylon Health’s 21.5(±9.63) screens (p<0.001). e number of screens utilized to reach a l disposition decision (triage) was 10.0 (±2.33) forDoctor Ai, whereas Babylon Health utilized 21.5 (±9.63) screens (p<0.001). Additionally, Doctor Ai used on average 13.9(±6.54) Yes/No events to determine the l diagnosis compared to Babylon Health’s 62.3 (±31.55) Yes/No events (p<0.001). In conclusion, the hypothetico-deductive system can diagnose more quickly and provide more accurate triage decisions compared to a decision tree powered system. However, both systems combined perform as well as physicians. It is important to evaluate whether such AI can be utilized to tackle pandemics in the U.S. healthcare system and in any develop ing country healthcare systems facing dire circumstances due to a scarcity of trained physicians