A New York public school is under fire after internal emails showed administrators knew their facial recognition system could not identify Black faces and had other significant problems.
Through the Freedom of Information Law, Motherboard obtained hundreds of emails from experts expressing concerns about the Lockport City School District’s decision to spend $2.7 million on SN Technologies’ AEGIS face and weapons detection system. Lockport is 11% Black and about an hour away from Niagara Falls.
The school district was told by auditors and scientists who had tested the system that it misidentified Black men four times more often and Black women 16 times more often than white men. The emails also show SN Technologies has outright lied about how they performed on racial bias tests.
Trump wants to exclude undocumented immigrants from the Census
Despite these concerns and an ACLU-backed lawsuit filed by parents, Lockport City School District turned on the system in January. It has largely been rendered useless because of pandemic regulations forcing everyone to wear a mask inside.
But parents in the district, particularly Black families, have been incensed about the school’s decision to spend millions on a system that routinely misidentifies Black people and was found to have a persistent problem of misidentifying broom handles as guns.
Parents in the district told Motherboard that the danger caused by this inaccurate system is incalculable. According to Motherboard, the AEGIS system “begins a process to automatically alert police when it detects weapons or certain people on the district’s ‘hot list.'”
In an interview with Motherboard, Lockport parent Jim Shultz said he was at a loss in understanding how the benefits of the system outweigh the potential consequences.
“The police have said if they get a notification they’re going to treat it as a live shooter system, and you have a system that’s predisposed to make mistakes and misidentify people. The risk of an accident, the risk of something horrible happening because the system is structured the way it is, to me, is 1 million times higher than [the chance] that the cameras are going to prevent a real situation,” Shultz said.
Source: A NY School District Knowingly Spent $2.7 Million On Racially Biased Facial Recognition Software
Recent Comments