The applications of machine learning (ML) and artificial intelligence (AI) continue to grow each year as more resources are devoted to developing mathematical methodologies and computational hardware that expand the environments in which ML and AI can be used. The most direct interaction that people have with these technologies is with several popular consumer electronics devices and services: Amazon Alexa and Google Home both use AI for speech recognition and natural language processing, streaming services such as Spotify may use ML to generate recommendations for users based on prior media consumption, Amazon leverages ML to offer products in which it believes the user may have interest, and financial institutions utilize ML/AI to predict and detect fraudulent account activity.
Use of ML/AI in healthcare research to improve efficiencies and outcomes is growing as well. At AHRM we are working to tackle problems that may be unique in the healthcare realm, and one “non-engineering” issue that we will consider here is that the output from such ML/AI research is different in scope to the output in a conventional research project. Because of this, a shift in expectations are sometimes required when presenting ML/AI as a research option.
In a conventional research project, a hypothesis is proposed and a protocol is designed to test that hypothesis, including a specific set of statistical analyses that seek to answer whether the hypothesis is true or false within some range of uncertainty. On the other hand, the result of research using ML or AI for healthcare outcomes is typically a predictive or classification model that can be applied to other datasets to generate estimates for future outcomes. Models as outputs for healthcare research isn’t entirely new, as most people are familiar with regression methods that generate equation(s) to describe relationships between exposures and outcomes. However, where regression models start to lose effectiveness and power with many variables or complicated relationships that cannot be defined by explicit equations, ML/AI models can pick up at that point and move forward. At AHRM we often say that ML/AI picks up where conventional statistics lets off. Let’s explore a case study to clarify what this really means.
Suppose we are examining drugs used to treat heart disease in a cohort of patients. Speaking in general terms, with a conventional research project we would establish a hypothesis regarding some important endpoints, then isolate each therapy and perform statistical analyses for each therapy to reach the specified endpoints. We may also examine covariates or exposures across each therapy and perform analyses taking those into consideration. Then the endpoint results would be compared across the treatments and the extent to which the original hypothesis is correct would be determined. So the mean change in some biomarker under therapy A may be 40% greater than the mean change in that same biomarker under therapy B, when controlled for covariates, and if the statistical power and error associated with the change are within the prescribed limits we can say that the therapies achieve different outcomes.
On the other hand, research using ML/AI in a similar situation would take into account all therapies, relevant exposures, and desired endpoints to develop a mathematical model relating all of those aspects. Rather than a single comparison statistic between therapies, we would have a model that describes the likely outcome for many combinations of therapy and covariates. Thus, we would be able to estimate the likely outcomes for patients on therapy A or B and any combination of covariates T, U, V, X, Y, and Z and so on. Because of this, ML/AI research may not necessarily provide a direct comparison between therapies and while there are techniques that can be used with the model to move towards some kind of comparison, if a direct comparison is desired then in many cases it is better to just utilize a conventional hypothesis-driven investigation.
Those who are familiar with regression models may be able to make the jump to ML/AI more easily, but regardless of research experience this difference in expectation can prove difficult for regulatory agencies, sponsors, IRBs, and decision-makers alike. A co-operative relationship between AHRM and the University at Buffalo Center for Computational Research not only provides nearly 4000 cores and 72 teraflops of supercomputing power for data-intensive ML/AI projects, but also serves as an extension of AHRM’s own knowledge and experience in navigating issues related to ML/AI research projects. Furthermore, at AHRM we also understand that although ML/AI is one of the more revolutionary technologies we have at our disposal, it is not a magic bullet and is not a replacement for other research methods. Nonetheless AHRM sees machine learning and artificial intelligence as an integral part of the future of healthcare and are committed to investigating how and why to use these technologies for healthcare outcomes research.
For further information or discussion, please contact:
Christopher Tyson, Ph.D.
+1-716-881-7565