Two AI systems match or beat doctors in Nature studies

AI Medical Agents Match or Outperform Physicians in Two Nature Studies

Two AI systems designed for clinical medicine performed at or above the level of physicians in simulated settings, according to a pair of studies published Wednesday in Nature. The results mark a new benchmark for medical AI but come with strong caveats: neither system has been tested with real patients in live clinical environments.

MIRA Operates Inside the Health Record

MIRA, developed by a team led by researchers at the Else Kröner Fresenius Center for Digital Health, is an autonomous AI agent that operates within a simulated electronic health record environment. Unlike previous medical AI systems that merely answer questions, MIRA can conduct patient interviews, order diagnostic tests, prescribe medications, and recommend hospital admissions.nature

Evaluated on hundreds of real emergency department cases, MIRA outperformed a panel of physicians in diagnostic accuracy, though experts noted that the AI ordered roughly twice as many blood tests as doctors did. Dr. Wei Xing, an assistant professor at the University of Sheffield, cautioned that MIRA’s advantage “is mostly driven by conditions with clear test results, like appendicitis and pancreatitis,” while for common conditions like pneumonia and urinary tract infections “the gap between them was smallest”.sciencemediacentre

Google’s AMIE Tackles Long-Term Care

The second study evaluated AMIE, Google’s Articulate Medical Intelligence Explorer, on the more complex task of managing chronic conditions across multiple visits. AMIE was compared to 21 primary care physicians across 100 multi-visit scenarios based on UK clinical guidelines. The system matched clinicians in overall management reasoning and scored higher in plan preciseness and guideline alignment.blog

Built on the long-context capabilities of Gemini models, AMIE pairs an empathetic dialogue agent with a management reasoning system that cross-references hundreds of pages of clinical knowledge. However, Alfonso Valencia, director of Life Sciences at the Barcelona Supercomputing Centre, noted that unlike MIRA, AMIE is not open-source, making independent evaluation impossible.sciencemediacentre

Experts Urge Caution on Real-World Use

Researchers and independent commentators uniformly stressed that both systems remain far from clinical deployment. “These technologies are not yet ready for autonomous use in clinical practice,” said Ignacio Miranda Gómez of the International Breast Cancer Centre in Barcelona. “The studies were conducted in controlled environments with simulated patients, so their efficacy and safety still need to be demonstrated in real hospitals and clinics”.sciencemediacentre

Both systems are text-based, meaning they cannot perform physical examinations or interpret tone of voice and body language — elements central to real clinical encounters. Prof. Catherine Pope of the University of Oxford noted that while the systems “can mimic some aspects of experienced physician performance,” much more research is needed, and “these technologies are unlikely to replace doctors”.sciencemediacentre

Google said it has launched a nationwide study to assess AMIE in real-world virtual care settings.blog