As people put more and more of their faith into Technology and Systems they don’t understand, what happens when the systems upon which you’ve put your faith are compromised? That’s what AI Security expert Dawn Song wonders.
Artificial intelligence won’t revolutionize anything if hackers can mess with it.
Recommended for YouThat’s the warning from Dawn Song, a professor at UC Berkeley who specializes in studying the security risks involved with AI and machine learning.
Speaking at EmTech Digital, an event in San Francisco produced by MIT Technology Review, Song warned that new techniques for probing and manipulating machine-learning systems—known in the field as “adversarial machine learning” methods—could cause big problems for anyone looking to harness the power of AI in business.
Song said adversarial machine learning could be used to attack just about any system built on the technology.
“It’s a big problem,” she told the audience. “We need to come together to fix it.”
Adversarial machine learning involves experimentally feeding input into an algorithm to reveal the information it has been trained on, or distorting input in a way that causes the system to misbehave. By inputting lots of images into a computer vision algorithm, for example, it is possible to reverse-engineer its functioning and ensure certain kinds of outputs, including incorrect ones.
Song presented several examples of adversarial-learning trickery that her research group has explored.
One project, conducted in collaboration with Google, involved probing machine-learning algorithms trained to generate automatic responses from e-mail messages (in this case the Enron e-mail data set). The effort showed that by creating the right messages, it is possible to have the machine model spit out sensitive data such as credit card numbers. The findings were used by Google to prevent Smart Compose, the tool that auto-generates text in Gmail, from being exploited.
Another project involved modifying road signs with a few innocuous-looking stickers to fool the computer vision systems used in many vehicles. In a video demo, Song showed how the car could be tricked into thinking that a stop sign actually says the speed limit is 45 miles per hour. This could be a huge problem for an automated driving system that relies on such information.
The tendency for people to take a Utopian approach in removing the human element from everything that makes us human is one of the more dangerous tendencies in which our society engages. Algorithms can be hacked just like databases and web servers. Whatever security we can invent will eventually fall prey to people who seek to destroy and/or take advantage of others. 