The Third Magic: AI

I like this definition of science from this Noah Smith article: in essence, the ability to come up with a generalizable & simple prediction model. For example, to calculate where an artillery shell will fall, we go back to Newton’s laws of physics and its consistently predictable. However, for messier things than the natural world like human language, there aren’t simple models and thus people argue that AI is the only way: take masses of data to form unintelligble models.

A big knock on AI is that because it doesn’t really let you understand the things you’re predicting, it’s unscientific. And in a formal sense, I think this is true. But instead of spending our effort on a neverending (and probably fruitless) quest to make AI fully interpretable, I think we should recognize that science is only one possible tool for predicting and controlling the world. Compared to science, black-box prediction has both strengths and weaknesses.

https://open.substack.com/pub/noahpinion/p/the-third-magic?r=q167&utm_campaign=post&utm_medium=email

The challenge, as this article points out, is what are the implications if we can’t understand the models? How good are our predictions really? Or, more importantly, how can we know those predictions are off? ChatGPT is a good example of this as it sound authoritative (and often is) but its not easy to know if its accurate.

Can data poisoning thwart face recognition systems?

Application of data poisoning for ML face recognition systems. Interesting approach. Could this be practically scaled for seamless use by everyone?

Fawkes isn’t intended to keep a facial recognition system like Facebook’s from recognizing someone in a single photo. It’s trying to more broadly corrupt facial recognition systems, performing an algorithmic attack called data poisoning. The researchers said that, ideally, people would start cloaking all the images they uploaded. That would mean a company like Clearview that scrapes those photos wouldn’t be able to create a functioning database, because an unidentified photo of you from the real world wouldn’t match the template of you that Clearview would have built over time from your online photos.

https://www.nytimes.com/2020/08/03/technology/fawkes-tool-protects-photos-from-facial-recognition.html

However, is it already too late to do via technical means and only legislation can regulate this?

But Clearview’s chief executive, Hoan Ton-That, ran a version of my Facebook experiment on the Clearview app and said the technology did not interfere with his system. In fact, he said, his company could use images cloaked by Fawkes to improve its ability to make sense of altered images. “There are billions of unmodified photos on the internet, all on different domain names,” Mr. Ton-That said. “In practice, it’s almost certainly too late to perfect a technology like Fawkes and deploy it at scale.”

https://www.nytimes.com/2020/08/03/technology/fawkes-tool-protects-photos-from-facial-recognition.html

AI researchers propose ‘bias bounties’ to put ethics principles…

Finding bias in machine learning models is a wholly different pursuit than bugs in rules-based code. While incentives and “red teaming” proposals follow a familiar model, this should follow more along the lines of adversarial ML. However, since most models are not exposed publicly, owners would have to agree to expose them opening up additional hurdles.

Researchers from Google Brain, Intel, OpenAI, and top research labs in the U.S. and Europe joined forces this week to release what the group calls a toolbox for turning AI ethics principles into practice. The kit for organizations creating AI models includes the idea of paying developers for finding bias in AI, akin to the bug bounties offered in security software. This recommendation and other ideas for ensuring AI is made with public trust and societal well-being in mind were detailed in a preprint paper published this week. The bug bounty hunting community might be too small to create strong assurances, but developers could still unearth more bias than is revealed by measures in place today, the authors say.

https://venturebeat.com/2020/04/17/ai-researchers-propose-bias-bounties-to-put-ethics-principles-into-practice/

An Algorithm That Grants Freedom, or Takes It Away

This NYT article on how algorithms are being used for various predictions in the justice system is insightful but hardly surprising.

In Philadelphia, an algorithm created by a professor at the University of Pennsylvania has helped dictate the experience of probationers for at least five years. The algorithm is one of many making decisions about people’s lives in the United States and Europe. Local authorities use so-called predictive algorithms to set police patrols, prison sentences and probation rules. In the Netherlands, an algorithm flagged welfare fraud risks. A British city rates which teenagers are most likely to become criminals.

https://www.nytimes.com/2020/02/06/technology/predictive-algorithms-crime.html

The problem here isn’t that algorithms are being used in these ways. In fact, its likely better to remove the frailties and individual biases at play often in these life changing decisions. However, there are two interelated issues here:

1. Lack of transparency breeds conspiracy. Its easy to claim that a heartless algorithm is wrong in making these major decisions but the bigger issue is that we don’t know how, by whom and with what training data they were developed.
2. Biases will be built in. We like to think algorithms don’t have biases but they were trained on human data. Humans have biases. There are systematic ways to work those out of machine learning algorithms but that’s a non-trivial effort. Per #1, unless we have transparency, we cannot make these better.

Governments should open these up (with proper privacy protections) to researchers who can discover and fix the biases. Only then will we trust them.