AI in Healthcare Is Dangerous Without Data Integrity
Every AI tool assumes it knows who the patient is. Most organizations have never checked.
By Martin A. Cody, Senior Vice President of Sales and Growth, 4medica
Everyone wants to do AI in healthcare. I hear it in nearly every conversation I have with health system leaders, and the numbers back it up: the AMA found this year that more than four in five physicians now use AI in their work, more than double the rate in 2023.
Almost no one is talking about the data going in. (Except perhaps legal and compliance.)
We all know the old line: garbage in, garbage out. In most industries, garbage out means a bad forecast or a wasted campaign. In clinical AI, garbage out potentially harms patients and causes reputational damage. And the most dangerous garbage in healthcare isn’t a typo or a missing field. It’s a record attached to the wrong patient identity.
Here’s my position: deploying AI on top of unresolved patient identity isn’t innovation. It’s risk that compounds, at scale.
The question AI can’t ask
Every AI tool starts from one assumption it has absolutely no way to test: that the record in front of it accurately belongs to one person, and only one person.
That assumption fails more often than most leaders want to admit. My name is two first names, and hotels regularly can’t find my reservation until someone flips them. Now multiply that by millions of patients, maiden names, nicknames and transposed birthdates. Duplicates split one patient across several records. Overlays merge two patients into one. For any health system of real size, the question isn’t whether you have patient identity risk. It’s how large is it and when will it surface?
AI can’t tell the difference. It produces a polished, confident answer either way. That’s why ECRI ranked AI in diagnosis the top patient safety concern of 2026, warning that these tools are only as reliable as the algorithms and data behind them.
Where today’s most-used AI meets bad identity
Nearly every high-volume AI application in healthcare reads from a patient record, writes to one, or creates one. Here are some examples of where risk compounds:
Clinical documentation. Ambient scribes are spreading faster than almost any technology I’ve seen in this industry, and more of them now draw from the chart for context. If that chart is a duplicate, the note is built on half a history. If it’s an overlay, another patient’s details can slip in. Then someone signs it. (Legal, is your pulse quickening yet?)
Chart summarization. AI summaries are now used millions of times a month to prep clinicians for visits. Summarize a split record and you miss everything on the other record. Summarize an overlay and you blend two people into one story.
Predictive and risk models. Most hospitals use models to forecast how inpatients will do and flag high-risk patients for follow-up. A split record can make a sick patient look healthy. An incorrectly merged record scores someone who doesn’t exist.
Patient access and scheduling. One in five large health systems now use AI for patient access, and voice agents are increasingly scheduling and pre-registering patients. This is where identity gets created. If an automated tool misses an existing record, it can create duplicates as fast as it books appointments.
Patient messaging. Nearly one in five physicians use AI to draft portal replies. Pull from an overlaid chart, and one patient’s information can land in another’s inbox.
Revenue cycle. At a recent industry conference, panelist after panelist described the same cat-and-mouse game: hospitals deploying AI to keep up with payers using AI to deny claims, and spending tens of thousands of hours to collect what they’re contractually owed. A hospital CFO and an industry consultant I spoke with agreed that won’t change while one side is driven by profit and the other by care. In that fight, identity errors are ammunition. Every mismatched name or birthdate is an easy denial, and AI working from the wrong record can create them at machine speed.
Why AI turns identity errors into a volume problem
Identity errors used to be local. One bad registration affected one patient, and someone might eventually catch it. AI changes that in three ways.
- It amplifies what’s already there. A duplicate patient record once touched occasionally is now read by summaries, models and drafting tools every day.
- It creates more. Automation at the front door can create records faster than any registration team, and documentation tools write to whatever record they’re pointed at.
- It compounds them. The scribe’s note becomes part of the chart. The summary reads the note. The risk model scores the same record. Orders and outreach follow the score.
Wrong patient. Wrong prediction. Wrong intervention.
And because AI-generated content is saved back to the record, the error doesn’t just pass through. It becomes part of the chart the next tool trusts.
We’re removing the safety net
On the first episode of our Dirty Data Done Right podcast, “AI’s Dirty Data Dilemma,” my guest John Lynn of Healthcare IT Today made a point I can’t shake. Clinicians have been quietly protecting us from bad data for years. When something looks off, they reorder the lab or repeat the scan. His worry is that as clinicians lean on AI, they’ll believe they have the whole picture when they don’t.
He’s right. A clean AI summary doesn’t look incomplete. It doesn’t look like it belongs to someone else. Healthcare doesn’t have a shortage of data. It has a shortage of data it can confidently trust, and AI widens that gap while hiding it.
People, process, then tech
The other message I heard from speaker after speaker at that conference, many of them leaders in AI, IT, revenue cycle and finance, was this: people, process, then tech, in that order. Great technology only takes you so far.
Identity proves the point. Duplicates are created by rushed registrations, inherited systems and acquisitions nobody cleaned up. No AI tool fixes that on its own. Before your next deployment:
- Know your numbers. If you can’t state your duplicate and overlay rates, you can’t say your AI is safe.
- Resolve identity first. Clean it before the scribe writes into it and the summary reads from it.
- Invest in the people at the front door. Registrars, HIM teams and data stewards protect the record every AI tool depends on. Give them the process and matching tools to do it well.
- Make identity an AI governance question. Alongside bias and accuracy, ask whether the model is looking at the right patient.
My fear is that many organizations won’t take this seriously until something goes badly wrong. And by badly wrong I mean camera crews at your facility because a patient has died due to someone acting on data they thought was accurate. It shouldn’t take that.
AI is an amplifier. Point it at clean, correctly matched data and it amplifies good care. Point it at duplicates and overlays and it amplifies harm, confidently and at scale. Before you deploy anything else, ask one question: does every record this tool touches belong to the right patient?
One patient. One record.
Book a Duplicate Reduction Strategy Session.
A 30-minute working session will help you identify where your duplicate rate really sits, what it’s costing you, and the fastest path to near-perfect.
