In a disturbing incident that highlights the potential pitfalls of AI-powered security systems, a Baltimore County high school student found himself in a nightmarish situation after an artificial intelligence system mistook his Doritos bag for a firearm.

Taki Allen, a student at Kenwood High School, experienced first-hand the unintended consequences of overzealous technology when he was forced to kneel and was handcuffed by school authorities after the AI security system raised an alarm. “I was just holding a Doritos bag , it was two hands and one finger out, and they said it looked like a gun,” Allen told local media.

The incident unfolded when the school’s security system flagged Allen’s snack as a potential weapon. Principal Katie Smith, who initially didn’t realize the alert had been canceled, reported the situation to the school resource officer, who subsequently contacted local police.

Omnilert, the company behind the AI gun detection system, offered a tepid response to the incident. While expressing regret, they bizarrely claimed that “the process functioned as intended” , a statement that raises serious questions about the system’s accuracy and the potential for traumatizing false positives.

This event underscores the growing concerns around AI technology in sensitive environments like schools, where false interpretations can lead to significant psychological and potentially dangerous physical consequences for students. The incident highlights the urgent need for more nuanced, human-centered approaches to school safety that prioritize student dignity and mental well-being.

As schools increasingly adopt advanced technologies for security purposes, incidents like these serve as critical reminders that algorithmic decision-making cannot replace human judgment, especially when dealing with young people in high-stress situations.