ChatGPT Beats Physicians in Patient Empathy

Artificial intelligence (AI) assistants like ChatGPT have the potential to revolutionize healthcare by providing accurate and empathetic responses to patients’ questions, thus reducing the burden on physicians. A new study published in JAMA Internal Medicine provides an early glimpse into the role that AI assistants could play in medicine by comparing written responses from physicians and those from ChatGPT to real-world health questions. The results of the study showed that a panel of licensed healthcare professionals preferred ChatGPT’s responses 79% of the time and rated them as higher quality and more empathetic than those from physicians.

The research team set out to answer the question of whether ChatGPT can respond accurately to questions patients send to their doctors. The team turned to social media where millions of patients publicly post medical questions to which doctors respond: Reddit’s AskDocs. The team randomly sampled 195 exchanges from AskDocs where a verified physician responded to a public question. The team provided the original question to ChatGPT and asked it to author a response. A panel of three licensed healthcare professionals assessed each question and the corresponding responses and were blinded to whether the response originated from a physician or ChatGPT. They compared responses based on information quality and empathy, noting which one they preferred.

The panel of healthcare professional evaluators preferred ChatGPT responses to physician responses 79% of the time. Additionally, ChatGPT responses were rated significantly higher in quality than physician responses, and they were more empathetic. The study’s authors noted that the ultimate solution isn’t throwing physicians out altogether but rather harnessing ChatGPT to help physicians provide better and empathetic care.

Investments into AI assistant messaging could impact patient health and physician performance by improving workflow. Integrating AI assistants into healthcare messaging must be done in the context of a randomized controlled trial to judge how the use of AI assistants impacts outcomes for both physicians and patients. The study’s authors believe that AI-augmented care is the future of medicine and that the opportunities for improving healthcare with AI are massive.