It’s one thing to train an algorithm to produce headline-grabbing results on paper. It’s quite another to train it to improve care for patients in practice. STAT reporter Casey Ross explores the challenges health systems must address to close that crucial gap in the field of artificial intelligence, where the science of validating algorithms is still a work in progress.
It is work that involves accounting for impacts on costs, clinician routines, and the innumerable variables presented by patients’ needs and experiences. Ultimately it requires definitively answering the core question that most people in medicine are still asking: Will AI really help people once it is unleashed into a world.
The challenges of applying machine learning (A.I.) to medicine are great. Perhaps the greatest one is validating the answer to the question presented to the machine. The sheer number of articles when a Pubmed Search is done reveals that the topic must be broken down into specialties, such as ophthalmology, radiology (imaging), Some results can be seen here.
Challenges in generalization to new populations and settings
The majority of AI systems are far from achieving reliable generalisability, let alone clinical applicability, for most types of medical data. A brittle model may have blind spots that can produce particularly bad decisions. A generalization can be hard due to technical differences between sites (including differences in equipment, coding definitions, EHR systems, and laboratory equipment and assays) as well as variations in local clinical and administrative practices.
There is an infinite number of possible applications for machine learning. Those who write the code seem to have developed a deep learning algorithm(s) and neural network processing. It is interesting that the overall process has been assigned an andromorphic name, alluding to how the brain processes information. We ascribe a digital binary process for the human brain, which is fundamentally flawed. Our knowledge of brain function is narrowly defined by glucose metabolism, magnetic resonance imaging, blood flow, and crude information about regional brain functions.
Further complicating the process is the wide range of machine learning algorithms. Google AI Research Microsoft, Amazon, and many others are competing for the market.
Several applications and collaboration are occurring, some have failed, such as the IBM Watson-MD Anderson Clinic collaboration regarding oncology treatment regimens.
The state of the art is still primitive and only experience will bring out the full potential of machine learning.
Webinar: From computer to the clinic: The challenges of implementing artificial intelligence in practice - STAT: View our latest webinar:
