© Hedgie @HedgieMarkets
https://x.com/HedgieMarkets/status/2054694174989877396
Ontario’s Auditor General found that AI medical transcription tools used by doctors across the province produced fabricated information, missed clinical details, and captured incorrect drug prescriptions during government testing. Nine of 20 systems hallucinated, including AI-generated treatment recommendations doctors never made. Twelve captured wrong drug names, and 17 missed key details about patients’ mental health issues. Four vendors were approved despite not submitting third-party audit reports. The tools are already in active use across Ontario hospitals and clinics.
My Take
17 of 20 systems missing key mental health information is the number I cannot stop thinking about, because mental health is precisely where transcription errors do the most damage. A missed mention of suicidal ideation or a fabricated treatment plan in a psychiatric note is not a clerical error, it is the kind of thing that gets people hurt. Government officials saying doctors review every transcript is technically accurate and practically meaningless to me, because the entire selling point of these tools is saving physicians time, and a busier doctor is more likely to skim a plausible-looking transcript than catch a hallucinated recommendation buried in three pages of notes.
This is the same dynamic I covered when Joe Riley’s father died after refusing cancer treatment based on a hallucinated Perplexity report. Medical AI is being deployed into clinical settings where the cost of an error is measured in patient outcomes, and procurement is being run by people treating it like any other software contract. Four vendors approved without submitting basic audit documentation shows me the guardrails are not where they need to be. Medical AI will produce a wave of harm cases over the next two to three years that forces a regulatory reset, and the patients absorbing the cost in the meantime never agreed to be part of the testing phase.
Hedgie