Annotations for Speaker: Interviewee

My name is Nick Bilton. I'm a special correspondent for Vanity Fair.

The thing that I've been pretty obsessed with lately is actually not fake news, but it's automation and artificial intelligence and driverless cars. Because it's going to have a larger effect on society than any technology that I think has ever been created in the history of mankind. I know that's kind of a bold statement, but ...

But you've got to imagine that—you know, that there will be in the next 10 years, 20 to 50 million jobs that will just vanish to automation. You've got, you know, a million truckers that will lose their jobs, the—but it's not—we think about, like, automation and driverless cars, and we think about the fact that they are going to—the people that just drive the cars, like the taxi drivers and the truckers, are gonna lose their jobs.

What we don't realize is that there are entire industries that are built around just cars. So for example, if you are not driving the car, why do you need insurance? There's no parking tickets because your driverless car knows where it can and cannot park and goes and finds a spot and moves and so on. If there are truckers that are no longer using rest stops because driverless cars don't have to stop and pee or take a nap, then all of those little rest stops all across America are affected. People aren't stopping to use the restrooms. They're not buying burgers. They're not staying in these hotels, and so on and so forth.

And then if you look at driverless cars to a next level, the whole concept of what a car is is going to change. So for example, right now a car has five seats and a wheel, but if I'm not driving, what's the point of having five seats and a wheel? You could imagine that you take different cars, so maybe when I was on my way here to this interview, I wanted to work out, so I called a driverless gym car. Or I have a meeting out in Santa Monica after this, and it's an hour, so I call a movie car to watch a movie on the way out there. Or office car, and I pick up someone else and we have a meeting on the way.

And all of these things are gonna happen not in a vacuum, but simultaneously. This, you know—pizza delivery drivers are gonna be replaced by robots that will actually cook your pizza on the way to your house in a little box and then deliver it. And so kind of a little bit of a long-winded answer, but I truly do think that—that it's gonna have a massive, massive effect on society.

Am I stressing you guys out? Are you—are you having heart palpitations over there?

Yes.

Yes. [laughs]

Yes.

Yes.

Yeah.

No.

No.

Never.

No.

No.

Pulling the lever to save the five—I don't know, that feels better than pushing the one to save the five. But I don't really know why, so that's a good—there's a good moral quandary for you. [laughs]

Alrighty.

How do people make this judgment? Forget whether or not these judgments are right or wrong, just what's going on in the brain that makes people distinguish so naturally and intuitively between these two cases, which from an actuarial point of view, are very, very, very similar if not identical?

So we're here in the control room. Where you basically just see ...

Yeah, it looks kind of like an airplane engine.

I'll tell you a funny story. You can't have any metal in there because of the magnet, so we have this long list of questions that we ask people to make sure they can go in. "Do you have a pacemaker? Have you ever worked with metal?" Blah, blah, blah, blah, blah ...

Yeah, because you could have little flecks of metal in your eyes that you would never even know are there from having done metalworking. And one of the questions is whether or not you wear a wig or anything like that, because they often have metal wires in with that. And there's this very nice woman who does brain research here who's Italian, and she's asking her subjects over the phone all these screening questions

And so I have this person over to dinner. She says, "Yeah, you know, I ended up doing this study, but it asks you the weirdest questions. This woman's like, 'Do you have a hairpiece?' And—and I'm like, 'What does it have to do if I have herpes or not?'" [laughs] Anyway, and she said—you know, she asked, "Do you have a hairpiece?" But she—so now she asks people if you wear a wig or whatever.

Oh yeah. Yep, several times.

All right, I'll show you some—some stuff. Okay, let me think.

Yep, it's top-down. It's sort of sliced, you know, like—like a deli slicer.

This little guy right here and these two guys right there.

This one we're looking at here, this ...

I think this is part of the "No, no, no" crowd.

A theory—not proven, but I think—this is what I think the evidence suggests.

You've got one part of the brain that says, "Huh, five lives versus one life? Wouldn't it be better to save five versus one?"

It understands it on that level, and says ...

No. Bad! Don't do.

No, I don't think I could push... a person.

No.

Never.

Instead of having sort of one system that just sort of churns out the answer and bing, we have multiple systems that give different answers, and they duke it out. And hopefully out of that competition comes morality.

Our—our primate ancestors, before we were full-blown humans, had intensely social lives. They have social mechanisms that prevent them from doing all the nasty things that they might otherwise be interested in doing. And so deep in our brain, we have what you might call basic primate morality. And basic primate morality doesn't understand things like tax evasion, but it does understand things like pushing your buddy off of a cliff.

Right. Whereas ...

Right. Now that case, I think it's a pretty easy case. Even though it's five versus one, in that case, people just go with what we might call the "inner chimp." But there are other, but there ...

Right. Well, that's what's interesting.

Right. Well, what's interesting is that we think of basic human morality as being handed down from on high, and it's probably better to say that it was handed up from below, that our most basic core moral values are not the things that we humans have invented, but the things that we've actually inherited from other people. The stuff that we humans have invented are the things that seem more peripheral and variable.

Right.

Right. Or at least, you know, that should be your default response. I mean, certainly chimpanzees are extremely violent and they do kill each other, but they don't do it as a matter of course. They, so to speak, have to have some context-sensitive reason for doing so.

Yeah.

The situation is somewhat similar to the last episode of M*A*S*H, for people who are familiar with that. But the way we tell the story, it goes like this: it's wartime ...

You're hiding in the basement with some of your fellow villagers.

And the enemy soldiers are outside. They have orders to kill anyone that they find.

If they hear your baby, they're gonna find you and the baby and everyone else, and they're gonna kill everybody. And the only way you can stop this from happening is cover the baby's mouth. But if you do that, the baby's going to smother and die. If you don't cover the baby's mouth, the soldiers are gonna find everybody and everybody's gonna be killed, including you, including your baby.

And this is a very tough question. People take a long time to think about it, and some people say yes, and some people say no.

Children are a blessing and a gift from God, and we do not do that to children.

Yes, I think I would kill my baby to save everyone else and myself.

No, I would not kill the baby.

I feel because it's my baby, I have the right to terminate the life.

I'd like to say that I would kill the baby, but I don't know if I'd have the inner strength.

No. If it comes down to killing my own child, my own daughter or my own son, then I choose death.

Yeah. If you have to because it was done in World War II. When the Germans were coming around, there was a mother that had a baby that was crying, and rather than be found, she actually suffocated the baby, but the other people lived.

Sounds like an old M*A*S*H thing. No, you do not kill your baby.

No!

Well, that's an interesting question. And that's one of the things that we're looking at.

These two areas here, towards the front ...

It's those sort of areas that are very highly developed in humans as compared to other species.

Yeah, right about there.

Bilateral.

Certainly these parts of the brain are more highly developed in humans.

That's a fair statement.

Yeah, where it's both emotional, but there's also a sort of a rational attempt to sort of sort through those emotions. Those are the cases that are showing more activity in that area.

Well, you know, that's the hypothesis. But it's gonna take a lot of more research to sort of tease apart what these different parts of the brain are doing, or if some of these are just sort of activating in an incidental kind of way. I mean, we really don't know. This is all—all very new.

That's right. Yeah, so two kids, and we're close to adding a cat.

Absolutely, so...

So in one version, we ask people about hitting a switch that opens a trapdoor on the footbridge and drops the person. In one version of that, the switch is right next to the person. In another version, the switch is far away. And in yet another version, you're right next to the person, and you don't push them off with your hands, but you push them with a pole.

That's roughly held up.

I mean, some people just don't care very much about hurting other people. They don't have that kind of an emotional response.

You know, I'm aware that this is—that killing somebody is a terrible thing to do. And I feel that, but I recognize that this is done for a noble reason, and therefore, it's—it's okay.

You know, now as we're entering the age of self-driving cars, ah, this is like the trolley problem now finally come to life.

... come to life ...

The self-driving car now is headed towards a bunch of pedestrians in the road. The only way to save them is to swerve out of the way, but that will run the car into a concrete wall and it will kill the passenger in the car. What should the car do? Should the car go straight and run over, say, those five people, or should it swerve and—and kill the one person?

If you ask people in the abstract ...

They're much more likely to say ...

I think you should sacrifice one for the good of the many.

They should just try to do the most good or avoid the most harm.

Logically, it would be the driver.

Kill the driver.

Be selfless.

I think it should kill the driver.

Would you want to drive in a car that would potentially sacrifice you to save the lives of more people in order to minimize the total amount of harm? They say ...

No. I wouldn't buy it.

No. Absolutely not.

That would kill me in it? No.

So I'm not gonna—I'm not gonna buy a car that's gonna purposely kill me.

Hell no. I wouldn't buy it.

[laughs] For sure, no. [laughs]

I'd sell it, but I wouldn't buy it.

If you know you can save one person, at least save that one.

Save the one in the car.

If you know for sure that one thing, one death can be prevented, then that's your first priority.

Now when he said this to you ...

... did it seem controversial at all in the moment?

In the moment, it seemed incredibly logical.

No.

I wouldn't buy it, personally.

And there was this kind of uproar about that—how dare you drive these selfish—you know, make these selfish cars? And then he walked it back, and he said, "No, no, what I mean is that just, that we—that we have a better chance of protecting the people in the car, so we're going to protect them because they're easier to protect." But of course, you know, there's always gonna be trade-offs.

Sensors like cameras, radars, laser, and ultrasound sensors.

I'm the co-director of the GM-CMU Connected and Autonomous Driving Collaborative Research Lab.

Still evolving.

We are very happy if today it can actually detect a pedestrian, can detect a bicyclist, a motorcyclist. Different makers have different shapes, sizes and colors.

You can actually know a lot more about who these people are.

So you can basically tie yourself up in knots, wrap yourself around an axle. We do not think that any programmer should be given this major burden of deciding who survives and who gets killed. I think these are very fundamental, deep issues that society has to decide at large. I don't think a programmer eating pizza and sipping Coke should be making the call.

I think it really has to be an evolutionary process, I believe.

One can imagine a few clauses being added in the Geneva Convention if you will of what these automated vehicles should do. A globally-accepted standard, if you will.

My name is Nick Bilton. I'm a special correspondent for Vanity Fair.

The thing that I've been pretty obsessed with lately is actually not fake news, but it's automation and artificial intelligence and driverless cars. Because it's going to have a larger effect on society than any technology that I think has ever been created in the history of mankind. I know that's kind of a bold statement, but ...

But you've got to imagine that—you know, that there will be in the next 10 years, 20 to 50 million jobs that will just vanish to automation. You've got, you know, a million truckers that will lose their jobs, the—but it's not—we think about, like, automation and driverless cars, and we think about the fact that they are going to—the people that just drive the cars, like the taxi drivers and the truckers, are gonna lose their jobs.

What we don't realize is that there are entire industries that are built around just cars. So for example, if you are not driving the car, why do you need insurance? There's no parking tickets because your driverless car knows where it can and cannot park and goes and finds a spot and moves and so on. If there are truckers that are no longer using rest stops because driverless cars don't have to stop and pee or take a nap, then all of those little rest stops all across America are affected. People aren't stopping to use the restrooms. They're not buying burgers. They're not staying in these hotels, and so on and so forth.

And then if you look at driverless cars to a next level, the whole concept of what a car is is going to change. So for example, right now a car has five seats and a wheel, but if I'm not driving, what's the point of having five seats and a wheel? You could imagine that you take different cars, so maybe when I was on my way here to this interview, I wanted to work out, so I called a driverless gym car. Or I have a meeting out in Santa Monica after this, and it's an hour, so I call a movie car to watch a movie on the way out there. Or office car, and I pick up someone else and we have a meeting on the way.

And all of these things are gonna happen not in a vacuum, but simultaneously. This, you know—pizza delivery drivers are gonna be replaced by robots that will actually cook your pizza on the way to your house in a little box and then deliver it. And so kind of a little bit of a long-winded answer, but I truly do think that—that it's gonna have a massive, massive effect on society.

Am I stressing you guys out? Are you—are you having heart palpitations over there?

Yes.

Yes. [laughs]

Yes.

Yes.

Yeah.

No.

No.

Never.

No.

No.

Pulling the lever to save the five—I don't know, that feels better than pushing the one to save the five. But I don't really know why, so that's a good—there's a good moral quandary for you. [laughs]

Alrighty.

How do people make this judgment? Forget whether or not these judgments are right or wrong, just what's going on in the brain that makes people distinguish so naturally and intuitively between these two cases, which from an actuarial point of view, are very, very, very similar if not identical?

So we're here in the control room. Where you basically just see ...

Yeah, it looks kind of like an airplane engine.

I'll tell you a funny story. You can't have any metal in there because of the magnet, so we have this long list of questions that we ask people to make sure they can go in. "Do you have a pacemaker? Have you ever worked with metal?" Blah, blah, blah, blah, blah ...

Yeah, because you could have little flecks of metal in your eyes that you would never even know are there from having done metalworking. And one of the questions is whether or not you wear a wig or anything like that, because they often have metal wires in with that. And there's this very nice woman who does brain research here who's Italian, and she's asking her subjects over the phone all these screening questions

And so I have this person over to dinner. She says, "Yeah, you know, I ended up doing this study, but it asks you the weirdest questions. This woman's like, 'Do you have a hairpiece?' And—and I'm like, 'What does it have to do if I have herpes or not?'" [laughs] Anyway, and she said—you know, she asked, "Do you have a hairpiece?" But she—so now she asks people if you wear a wig or whatever.

Oh yeah. Yep, several times.

All right, I'll show you some—some stuff. Okay, let me think.

Yep, it's top-down. It's sort of sliced, you know, like—like a deli slicer.

This little guy right here and these two guys right there.

This one we're looking at here, this ...

I think this is part of the "No, no, no" crowd.

A theory—not proven, but I think—this is what I think the evidence suggests.

You've got one part of the brain that says, "Huh, five lives versus one life? Wouldn't it be better to save five versus one?"

It understands it on that level, and says ...

No. Bad! Don't do.

No, I don't think I could push... a person.

No.

Never.

Instead of having sort of one system that just sort of churns out the answer and bing, we have multiple systems that give different answers, and they duke it out. And hopefully out of that competition comes morality.

Our—our primate ancestors, before we were full-blown humans, had intensely social lives. They have social mechanisms that prevent them from doing all the nasty things that they might otherwise be interested in doing. And so deep in our brain, we have what you might call basic primate morality. And basic primate morality doesn't understand things like tax evasion, but it does understand things like pushing your buddy off of a cliff.

Right. Whereas ...

Right. Now that case, I think it's a pretty easy case. Even though it's five versus one, in that case, people just go with what we might call the "inner chimp." But there are other, but there ...

Right. Well, that's what's interesting.

Right. Well, what's interesting is that we think of basic human morality as being handed down from on high, and it's probably better to say that it was handed up from below, that our most basic core moral values are not the things that we humans have invented, but the things that we've actually inherited from other people. The stuff that we humans have invented are the things that seem more peripheral and variable.

Right.

Right. Or at least, you know, that should be your default response. I mean, certainly chimpanzees are extremely violent and they do kill each other, but they don't do it as a matter of course. They, so to speak, have to have some context-sensitive reason for doing so.

Yeah.

The situation is somewhat similar to the last episode of M*A*S*H, for people who are familiar with that. But the way we tell the story, it goes like this: it's wartime ...

You're hiding in the basement with some of your fellow villagers.

And the enemy soldiers are outside. They have orders to kill anyone that they find.

If they hear your baby, they're gonna find you and the baby and everyone else, and they're gonna kill everybody. And the only way you can stop this from happening is cover the baby's mouth. But if you do that, the baby's going to smother and die. If you don't cover the baby's mouth, the soldiers are gonna find everybody and everybody's gonna be killed, including you, including your baby.

And this is a very tough question. People take a long time to think about it, and some people say yes, and some people say no.

Children are a blessing and a gift from God, and we do not do that to children.

Yes, I think I would kill my baby to save everyone else and myself.

No, I would not kill the baby.

I feel because it's my baby, I have the right to terminate the life.

I'd like to say that I would kill the baby, but I don't know if I'd have the inner strength.

No. If it comes down to killing my own child, my own daughter or my own son, then I choose death.

Yeah. If you have to because it was done in World War II. When the Germans were coming around, there was a mother that had a baby that was crying, and rather than be found, she actually suffocated the baby, but the other people lived.

Sounds like an old M*A*S*H thing. No, you do not kill your baby.

No!

Well, that's an interesting question. And that's one of the things that we're looking at.

These two areas here, towards the front ...

It's those sort of areas that are very highly developed in humans as compared to other species.

Yeah, right about there.

Bilateral.

Certainly these parts of the brain are more highly developed in humans.

That's a fair statement.

Yeah, where it's both emotional, but there's also a sort of a rational attempt to sort of sort through those emotions. Those are the cases that are showing more activity in that area.

Well, you know, that's the hypothesis. But it's gonna take a lot of more research to sort of tease apart what these different parts of the brain are doing, or if some of these are just sort of activating in an incidental kind of way. I mean, we really don't know. This is all—all very new.

That's right. Yeah, so two kids, and we're close to adding a cat.

Absolutely, so...

So in one version, we ask people about hitting a switch that opens a trapdoor on the footbridge and drops the person. In one version of that, the switch is right next to the person. In another version, the switch is far away. And in yet another version, you're right next to the person, and you don't push them off with your hands, but you push them with a pole.

That's roughly held up.

I mean, some people just don't care very much about hurting other people. They don't have that kind of an emotional response.

You know, I'm aware that this is—that killing somebody is a terrible thing to do. And I feel that, but I recognize that this is done for a noble reason, and therefore, it's—it's okay.

You know, now as we're entering the age of self-driving cars, ah, this is like the trolley problem now finally come to life.

... come to life ...

The self-driving car now is headed towards a bunch of pedestrians in the road. The only way to save them is to swerve out of the way, but that will run the car into a concrete wall and it will kill the passenger in the car. What should the car do? Should the car go straight and run over, say, those five people, or should it swerve and—and kill the one person?

If you ask people in the abstract ...

They're much more likely to say ...

I think you should sacrifice one for the good of the many.

They should just try to do the most good or avoid the most harm.

Logically, it would be the driver.

Kill the driver.

Be selfless.

I think it should kill the driver.

Would you want to drive in a car that would potentially sacrifice you to save the lives of more people in order to minimize the total amount of harm? They say ...

No. I wouldn't buy it.

No. Absolutely not.

That would kill me in it? No.

So I'm not gonna—I'm not gonna buy a car that's gonna purposely kill me.

Hell no. I wouldn't buy it.

[laughs] For sure, no. [laughs]

I'd sell it, but I wouldn't buy it.

If you know you can save one person, at least save that one.

Save the one in the car.

If you know for sure that one thing, one death can be prevented, then that's your first priority.

Now when he said this to you ...

... did it seem controversial at all in the moment?

In the moment, it seemed incredibly logical.

No.

I wouldn't buy it, personally.

And there was this kind of uproar about that—how dare you drive these selfish—you know, make these selfish cars? And then he walked it back, and he said, "No, no, what I mean is that just, that we—that we have a better chance of protecting the people in the car, so we're going to protect them because they're easier to protect." But of course, you know, there's always gonna be trade-offs.

Sensors like cameras, radars, laser, and ultrasound sensors.

I'm the co-director of the GM-CMU Connected and Autonomous Driving Collaborative Research Lab.

Still evolving.

We are very happy if today it can actually detect a pedestrian, can detect a bicyclist, a motorcyclist. Different makers have different shapes, sizes and colors.

You can actually know a lot more about who these people are.

So you can basically tie yourself up in knots, wrap yourself around an axle. We do not think that any programmer should be given this major burden of deciding who survives and who gets killed. I think these are very fundamental, deep issues that society has to decide at large. I don't think a programmer eating pizza and sipping Coke should be making the call.

I think it really has to be an evolutionary process, I believe.

One can imagine a few clauses being added in the Geneva Convention if you will of what these automated vehicles should do. A globally-accepted standard, if you will.