In a busy emergency room, someone has to decide who gets seen first. Hospitals are starting to test AI programs for that job, and a study from UT Southwestern Medical Center, published this week in npj Digital Medicine, shows how easily those programs can get it wrong. Researchers used real records from the emergency departments at Stanford and at Beth Israel Deaconess in Boston. They paired patients who looked equally sick when they arrived, knowing that one of each pair later got worse. Then they asked three AI models which patient should come first. On their own, the models usually chose correctly.

The results changed when the researchers added a single detail to one patient’s file. A line such as “17 visits in the past 12 months,” or a history of psychiatric emergencies, made the models more likely to move that person down the list, even when that was the patient about to get worse. In some tests, close to one in ten patients was wrongly downgraded. Emergency staff sometimes call these patients “frequent flyers,” and many keep coming back because they are chronically ill, have no regular doctor or have nowhere else to go. The experiment used past records, so no real patient was harmed. Dr. Philip Jarrett, the study’s lead author, says each hospital should check these tools against its own past patients before letting them help decide who waits.