Misplaced Faith in Technology

As people put more and more of their faith into Technology and Systems they don’t understand, what happens when the systems upon which you’ve put your faith are compromised? That’s what AI Security expert Dawn Song wonders.

Artificial intelligence won’t revolutionize anything if hackers can mess with it.
Recommended for You

That’s the warning from Dawn Song, a professor at UC Berkeley who specializes in studying the security risks involved with AI and machine learning.

Speaking at EmTech Digital, an event in San Francisco produced by MIT Technology Review, Song warned that new techniques for probing and manipulating machine-learning systems—known in the field as “adversarial machine learning” methods—could cause big problems for anyone looking to harness the power of AI in business.

Song said adversarial machine learning could be used to attack just about any system built on the technology.

“It’s a big problem,” she told the audience. “We need to come together to fix it.”

Adversarial machine learning involves experimentally feeding input into an algorithm to reveal the information it has been trained on, or distorting input in a way that causes the system to misbehave. By inputting lots of images into a computer vision algorithm, for example, it is possible to reverse-engineer its functioning and ensure certain kinds of outputs, including incorrect ones.

Song presented several examples of adversarial-learning trickery that her research group has explored.

One project, conducted in collaboration with Google, involved probing machine-learning algorithms trained to generate automatic responses from e-mail messages (in this case the Enron e-mail data set). The effort showed that by creating the right messages, it is possible to have the machine model spit out sensitive data such as credit card numbers. The findings were used by Google to prevent Smart Compose, the tool that auto-generates text in Gmail, from being exploited.

Another project involved modifying road signs with a few innocuous-looking stickers to fool the computer vision systems used in many vehicles. In a video demo, Song showed how the car could be tricked into thinking that a stop sign actually says the speed limit is 45 miles per hour. This could be a huge problem for an automated driving system that relies on such information.

The tendency for people to take a Utopian approach in removing the human element from everything that makes us human is one of the more dangerous tendencies in which our society engages. Algorithms can be hacked just like databases and web servers. Whatever security we can invent will eventually fall prey to people who seek to destroy and/or take advantage of others.

Something Might Happen

Dad said not to bring my glove. “We’re all the way up in the red seats,” he said.“No one’s gonna hit it up there.” Then, as if to emphasize the point, “No way. Not. At. All.”

But I brought it anyway.

It was an early April morning in 1988. The late ’80s were good years – after Pete Rose had broken the record but before the mess of banishment – when the Reds seemed to always finish second to the Dodgers, Giants, or Astros, no matter how hard they tried.

Dad and I rode a city bus down Winton Road from the northern suburbs, through St. Bernard, through Corryville, past UC, and straight through Over the Rhine like a Barry Larkin line drive, ending up on Fountain Square an hour ahead of the Findlay Market parade. It was Opening Day, the holiest of baseball holidays, and we reveled in our annual pilgrimage.

I held the glove under my left arm. Dad eyed me sideways. “You never know,” I said. “Something might happen.”

Dad got a coffee, I opened a bag of peanuts, and we sat on the steps overlooking Fifth Street. The crowds started to gather. “Tell me the Johnny Bench story again,” I said, and he launched into an elaborate tale of the time, when he was in middle school, he and a friend got a ride home from Crosley Field with the new Rookie catcher, Johnny Bench.

“He rolled up to us in a big convertible and said, “You boys need a lift? Of course we said yes, and he drove us all the way home.”

“What did you talk about?”

“Nothing. We was both too scared to say anything, so we sat in silence the whole way.”

The parade started. Marching bands, decorated Cadillacs carrying politicians, and elaborate floats with local celebrities carrying signs for hometown staples like Goldstar Chili and JTM hamburgers went by in quick succession. When the last float passed, the crowd flowed in behind, following the parade like a jubilant, New Orleans wake, down to Riverfront Stadium for the start of a brand new season.

We made it to our seats high up in the red seats about as far aqay feom the field as you could get.

“You think the Reds have a chance this year?” I asked.

“I don’t know,” Dad said. “Soto’s washed up, Bo Diaz is a rusted out, chain-link fence behind the plate, and I’m still not sure about Larkin over Stillwell at short.”

I soured a little. He noticed.

“Then again, that Tom Browning is pretty good, Eric Davis can hit the cover off the ball, and with Franco closing, you never know…”

“Something might happen,” we said together, and laughed. It was always easy to laugh when it was him and me.

The game was a tough one. Mario Soto gave up an early lead to the Cardinals. He was, indeed, washed up, which didn’t bode well for the season. The Reds were down 4-1 in the sixth, but battled back to a tie in the seventh and took it to extra innings.

“More baseball for the exact same price!” Dad used to say.

The game was fun, but I wanted a ball. I stood for most of the afternoon with my glove on my hand, ready. Dad just shook his head like he knew something I didn’t.

For a while, it seemed like that might be the case but, in the 11th, Tracey Jones tagged a foul shot off of a Cardinal reliever, and I watched as the ball soared up to our section.

“Go for it!”

I ran down the steps to the front row, reaching my glove as far out over the railing as I could, hoping for a miracle. The ball danced around the webbing at the tip of my glove, then bounced away, falling to the blue seats below. I returned to my seat, dejected.

“That’s alright,” Dad said. “You’ll get the next one.”

But the next one didn’t come. Not then, anyway. Kal Daniels knocked in a run in the the twelfth, “And this one belongs to the Reds!” everyone shouted, mimicking Marty Brennaman’s signature phrase. The Reds won 5-4 and, right then, everyone in the stands truly believed that day’s success would carry us all the way to the World Series.

Opening Day does that to you, somehow. It makes believers out of all of us.

I fell asleep on the bus afterward. Dad carried me from the bus stop tonour house, my glove secured safely in his left arm, the same way he carried his glove when he was young. It was a good day.

It was a good year, too. Chris Sabo got his start, and it wasn’t long before all the kids in my school wanted their own pair of Spuds McKenzie goggles. Danny Jackson won 23 games and would have won a Cy Young, too, if not for Orel Hershiser. Tom Browning threw a perfect game and Ron Robinson came within one strike of the same feat, too.

The Reds hosted the All Star Game, with Barry Larkin – who really WAS better than Kurt Stillwell it turned out – on the team, proving himself more than capable of carrying Concepcion’s mantle.

Dad and I went to a lot of games that year and in the years to follow. But nothing beats Opening Day. No matter how bad the Reds are, no matter how bleak the prospects, on Opening Day anything is possible. On Opening Day, the slate is clean and the entire season stretches out in front of you like a dream.

Another Opening Day is right around the corner. The Redlegs could tank, sure, but you never know. Magic could strike at any time. It happens. All it takes is the willingness to believe, just for a day. So grab your glove, keep your eyes open, and wait for something to happen.

Because on Opening Day, something always does.