More than two dozen AI experts are calling for Amazon to stop selling facial recognition software to law enforcement until legislation and oversight is put in place.
The letter, published today (April 3) and shared with Quartz, claims that Amazon’s general manager of AI, Matthew Wood, and vice president of global public policy Michael Punke misrepresented technical aspects of research that suggests Amazon’s facial recognition is less accurate on women and people of color in a company blog post.
“Overall, we find Dr. Wood and Mr. Punke’s response to the peer-reviewed research findings disappointing,” the letter says. “We hope that the company will instead thoroughly examine all of its products and question whether they should currently be used by police.”
The referenced research, called Gender Shades and published by MIT Media Lab’s Joy Buolamwini and Deborah Raji, found that Amazon’s Rekognition facial recognition software mistook women of color for men 31% of the time, while making no errors on classifying the gender of white men.
“There is a barrier between critiques of facial analysis technologies and the companies making them. It’s important for us to think about, for example, how gender is conceptualized and encoded into commercial facial analysis technologies that are available today,” Morgan Klaus Scheuerman, an information-science doctoral candidate at UC Boulder and letter signee, told Quartz via email.
“Facial analysis technologies, when they are packaged up as cloud-based services, can be appropriated for malicious intent. Even in ways that the companies supplying them aren’t even aware of,” he wrote. “Amazon has a lot of influence in the field of facial analysis technology. It is one of the major suppliers of these types of computer vision services and can help shape norms around its development and use.”
Those signing the petition also include a 2019 winner of computer science’s highest honor, the Turing Award, Yoshua Bengio, and former head scientist at Amazon Web Services, Anima Anandkumar, along with researchers from Google, Facebook, Microsoft, Alphabet’s DeepMind, Harvard, and the University of California, Berkeley.
“What we hope is for a broad dialogue,” Anandkumar told Quartz, who is now the director of research at chip manufacturer Nvidia, and noted that she was not speaking on behalf of her company. “How are we measuring these systems, and what are the different evaluation metrics we need to think about, and what impact is it going to have on society by putting it in the hands of law enforcement?”
She added that what convinced her to sign was the technical nature of the letter, and that having the scientific discussion about making facial recognition work is necessary before it can be legislated on effectively and put into real-world situations.
“We have to have the right version of regulations, one that doesn’t stop us pushing research forward, but at the same time that would be effective,” Anandkumar said. “It’s a complex problem, but I’m optimistic that with more open dialogue that can happen.”