AI scribe tools produce lower quality medical notes compared to human clinicians

A cross-sectional evaluation found that clinical notes generated by ambient artificial intelligence (AI) scribe tools were consistently lower in quality than those produced by human clinicians. AI scribes performed especially poorly on key elements needed for accurate medical documentation, including thoroughness, organization, and usefulness. The findings, published today in the Annals of Internal Medicine , were also presented by lead author Dr. Ashok Reddy , associate professor (General Internal Medicine) at the breaking news scientific plenary session at the 2026 American College of Physicians’ (ACP) Internal Medicine Meeting in San Francisco. Senior author is Dr. Karin Nelson. Ambient AI scribes have been shown to reduce administrative burden among physicians, but prior evaluations have not focused on documentation quality of these scribes. Researchers from the Veterans Health Administration sought to compare the quality of AI-generated clinical notes with that of human-written notes. Researchers used audio recordings of five standardized primary care visits and asked 11 AI scribe tools and 18 human clinicians, including physicians, pharmacists, and nurse care managers, to generate clinical notes from them. Thirty blinded raters scored each note using the modified Physician Documentation Quality Instrument (PDQI-9), which measures quality across 10 domains. In every case, human notes scored higher, in some cases dramatically so, and AI notes consistently scored lower across quality domains including accuracy, thoroughness, usefulness, organization, and comprehensiveness. The authors conclude that while AI scribes may help reduce documentation burden, rigorous and ongoing testing of their accuracy and quality is critical before relying on them in clinical care. For clinicians, AI scribes should be regarded as tools for generating draft documentation that requires review and editing, rather than as a substitute for clinician-authored notes.