Examine: ChatGPT gave ‘largely acceptable’ solutions to heart problems prevention questions

ChatGPT offered “largely acceptable” responses to questions on heart problems prevention, in keeping with a research letter published in JAMA. 

Researchers put collectively 25 questions on elementary ideas for stopping coronary heart illness, together with threat issue counseling, check outcomes and medicine data. They posed the inquiries to the AI chatbot 3 times and every set of responses was graded by a clinician. 

The clinician rated the responses as acceptable, inappropriate or unreliable if the chatbot’s responses various in every set. The reviewers additionally evaluated the responses in two contexts: first as a patient-facing platform after which as draft responses to affected person messages despatched to a clinician for evaluation. 

The examine discovered 21 of the 25 questions have been thought of acceptable in each contexts, whereas 4 have been graded inappropriate in each eventualities. In three of the 4 units of inappropriate responses, all three solutions have been incorrect, whereas one set had just one inappropriate response. 


The researchers famous a number of limitations of their evaluation. At the moment, the chatbot is not designed for medical use, and heart problems prevention cannot be coated in solely 25 questions. 

In future research, they stated researchers may use extra reviewers to guage responses or arrange a proper system for grading responses that did not rely as closely on a clinician’s subjective opinion. 

“Findings counsel the potential of interactive AI to help scientific workflows by augmenting affected person training and patient-clinician communication round frequent CVD prevention queries. For instance, such an software might present conversational responses to easy queries on informational platforms or create automated draft responses to affected person digital messages for clinicians,” they wrote. “Whether or not these approaches can enhance readability ought to be explored, as a result of prior work has indicated low readability of sure on-line affected person instructional supplies for CVD prevention.”


There are a selection of corporations touting chatbots for healthcare functions. Wysa, which makes a chatbot that guides customers by way of cognitive behavioral remedy for considerations like low temper, nervousness and stress, raised $20 million last summer

One other firm behind a psychological well being chatbot, Woebot Well being, scored a $9.5 million funding from Leaps by Bayer almost a yr in the past. In 2021, Woebot acquired FDA Breakthrough Device Designation for its postpartum melancholy digital therapeutic.

Symptom checking chatbot Ada Well being, which introduced a $30 million Series B extension in February 2022, not too long ago launched a digital COVID-19 care journey that was developed with Pfizer


Leave a Reply