Trolley problem, again

Posted 24 June 2016 by Matt Young

I ran across two articles today on the trolley problem as it applies to driverless (or self-driving) cars: one in Science by Joshua Greene and one in the LA Times by Karen Kaplan. Both are based on this article by Jean-François Bonnefon and colleagues in today's issue of Science. We discussed the trolley problem briefly here at PT last October. More precisely, we discussed an extended trolley problem wherein you are in a driverless car and the choices are to kill 5 people, kill 1 person, or kill yourself. The current research also concerns driverless cars. Not surprisingly, the researchers found support for driverless cars choosing to kill one person rather than five, but they also found that such support withered when you were the one. Their result in fact is completely consistent with the research of April Bleske-Rechek, which I outlined in my talk on the evolution of morality. Professor Bleske-Rechek found that people's willingness to sacrifice one person in favor of five decreased with, for example, increasing relatedness of the one person. Professor Bonnefon and his colleagues employed a survey, similarly to Professor Bleske-Rechek and hers, and found that people's enthusiasm for a "utilitarian" car – a car that will sacrifice the driver in favor of a larger number of pedestrians – decreased as the driver became closer related to the respondent. Professor Greene asks whether driverless cars should indeed be programmed to be utilitarian in that sense; or programmed to behave in some other way, say, to save the driver; or simply be programmed to avoid a crash, come what may. He notes,

Manufacturers of utilitarian cars will be criticized for their willingness to kill their own passengers. Manufacturers of cars that privilege their own passengers will be criticized for devaluing the lives of others and their willingness to cause additional deaths.

Professor Bonnefon and colleagues similarly conclude,

Although people tend to agree that everyone would be better off if AVs [autonomous vehicles] were utilitarian (in the sense of minimizing the number of casualties on the road), these same people have a personal incentive to ride in AVs that will protect them at all costs. Accordingly, if both self-protective and utilitarian AVs were allowed on the market, few people would be willing to ride in utilitarian AVs, even though they would prefer others to do so. ... [M]ost people seem to disapprove of a regulation that would enforce utilitarian AVs. Second--and a more serious problem--our results suggest that such regulation could substantially delay the adoption of AVs, which means that the lives saved by making AVs utilitarian may be outnumbered by the deaths caused by delaying the adoption of AVs altogether.

This question – whether to design utilitarian cars or to let the chips fall where they may – is precisely the trolley problem which, as I showed in my talk, is very real and not simply a philosophical exercise.

12 Comments

RJ · 26 June 2016

We could move to a society with a lot fewer cars instead, on the utilitarian grounds that car accidents are one of the leading preventable causes of death . The resulting massive reduction in the need for petroleum also would greatly reduce the likelihood of resource-based wars which bring death as well as other very disutilitous outcomes to people, 99% of the time people who have no stake in the outcome of said wars.

Wanna massively increase human utility? Massively decrease car travel.

I thank you for demonstrating that this sort of problem deserves to be taken seriously rather being used as a reason to ridicule philosophy. In my opinion, a refusal to consider the abstract entailments of one's views is a refusal to examine oneself.

Matt Young · 26 June 2016

I certainly think we need to reduce travel by car, but we are not going to do so any time soon. The next best approach to reducing "death by car" may therefore be to develop driverless cars (and also vastly smaller cars. In the 1970's, during the so-called oil crisis, we confidently predicted that by 2000 cars would weigh 500 kg, get 100 mi/gal, and bounce off each other when they collided. Almost as serious an error as when I predicted that the 3-point basket would have little effect on girls' basketball).

Back on task: I have been thinking more about the problem and realize that it is not the same as the trolley problem. In the trolley problem, you are guaranteed to kill 5 people or 1. In the driverless-car problem, you are not guaranteed to kill anyone, if the car can avoid an accident. So the car has to make a choice among perhaps 4 options: kill 5 people, kill 1, kill the driver, and avoid an accident entirely.

Such consideration, combined with the observation that hardly anyone would buy a car programmed to kill the driver, makes me think the best option is to program the car so that it does its best to avoid an accident, come what may. Supposedly, it is more capable of avoiding an accident than is a human driver, so maybe they should not be programmed to make any ethical decisions whatsoever.

I wonder what other people think?

Mike Elzinga · 26 June 2016

The Trolley Problem actually fits within a broader perspective of human choices that affect the lives of millions of others. If those "others" are strangers, or "foreigners," or especially those who are not born yet, then the weight of these others in critical decisions is considerably diminished in the decision-making process. Zero-sum games, such as "The Tragedy of the Commons," have this feature.

Economist, Nicholas Georgescu-Roegen, had an interesting way of putting it with regard to how our economic decisions about consumption affect future generations. He said that our decision-making processes suggest that we think, "Why should we care about posterity; what has posterity ever done for us?"

Physicist, Albert Bartlett's book "The Essential Exponential outlines the issue in mathematics that is relatively easy to understand.

The relationship of this to Atonomous Vehicles is that that AV problem fits into a larger context of population, life-style, working environments, communication, and the distribution of goods and services.

It doesn't appear to me that the AV problem has a satisfactory solution within the context of our current economic culture and life styles. We would have to revampt our entire world economy in order for AVs to be safe in a way that doesn't put our individual lives up against the lives of others of less personal value to us.

PaulBC · 26 June 2016

Mike Elzinga said: Economist, Nicholas Georgescu-Roegen, had an interesting way of putting it with regard to how our economic decisions about consumption affect future generations. He said that our decision-making processes suggest that we think, "Why should we care about posterity; what has posterity ever done for us?"

I think it was just the word "economist" that reminded of this Adam Smith quote https://en.wikiquote.org/wiki/Adam_Smith from Theory of Moral Sentiments.

Let us suppose that the great empire of China, with all its myriads of inhabitants, was suddenly swallowed up by an earthquake, and let us consider how a man of humanity in Europe, who had no sort of connection with that part of the world, would be affected upon receiving intelligence of this dreadful calamity. He would, I imagine, first of all, express very strongly his sorrow for the misfortune of that unhappy people, he would make many melancholy reflections upon the precariousness of human life, and the vanity of all the labours of man, which could thus be annihilated in a moment. He would too, perhaps, if he was a man of speculation, enter into many reasonings concerning the effects which this disaster might produce upon the commerce of Europe, and the trade and business of the world in general. And when all this fine philosophy was over, when all these humane sentiments had been once fairly expressed, he would pursue his business or his pleasure, take his repose or his diversion, with the same ease and tranquillity, as if no such accident had happened. The most frivolous disaster which could befall himself would occasion a more real disturbance. If he was to lose his little finger to-morrow, he would not sleep to-night; but, provided he never saw them, he will snore with the most profound security over the ruin of a hundred millions of his brethren, and the destruction of that immense multitude seems plainly an object less interesting to him, than this paltry misfortune of his own. To prevent, therefore, this paltry misfortune to himself, would a man of humanity be willing to sacrifice the lives of a hundred millions of his brethren, provided he had never seen them? Human nature startles with horror at the thought, and the world, in its greatest depravity and corruption, never produced such a villain as could be capable of entertaining it. But what makes this difference? When our passive feelings are almost always so sordid and so selfish, how comes it that our active principles should often be so generous and so noble? When we are always so much more deeply affected by whatever concerns ourselves, than by whatever concerns other men; what is it which prompts the generous, upon all occasions, and the mean upon many, to sacrifice their own interests to the greater interests of others? It is not the soft power of humanity, it is not that feeble spark of benevolence which Nature has lighted up in the human heart, that is thus capable of counteracting the strongest impulses of self-love. It is a stronger power, a more forcible motive, which exerts itself upon such occasions. It is reason, principle, conscience, the inhabitant of the breast, the man within, the great judge and arbiter of our conduct.

eric · 27 June 2016

RJ said: We could move to a society with a lot fewer cars instead...

Matt Young said: I certainly think we need to reduce travel by car, but we are not going to do so any time soon....

Actually, I thought one component of the 'driverless car' concept was more ridesharing. Sort of like the Uber app; you push a button on your phone to summon a car, you tell it where you are going, the system matches you to a car and sends it around to pick you up. Both could be incentivised by the use of specail driverless/commute lanes. So I expect the two concepts will go together; driverless cars and fewer cars on the road. On the original question; I don't particularly see the issue with 'selfish' programming here. For me, as long as everyone understands that that's how they work, programming them that way (i.e., to save their passengers in preference to others) doesn't pose a moral problem. IOW I'm okay with you riding in a car that will run me over to save you, if you are okay with vice versa.

TomS · 27 June 2016

eric said:
RJ said: We could move to a society with a lot fewer cars instead...

Matt Young said: I certainly think we need to reduce travel by car, but we are not going to do so any time soon....
Actually, I thought one component of the 'driverless car' concept was more ridesharing. Sort of like the Uber app; you push a button on your phone to summon a car, you tell it where you are going, the system matches you to a car and sends it around to pick you up. Both could be incentivised by the use of specail driverless/commute lanes. So I expect the two concepts will go together; driverless cars and fewer cars on the road. On the original question; I don't particularly see the issue with 'selfish' programming here. For me, as long as everyone understands that that's how they work, programming them that way (i.e., to save their passengers in preference to others) doesn't pose a moral problem. IOW I'm okay with you riding in a car that will run me over to save you, if you are okay with vice versa.

What about a car which takes you to where it thinks you ought to go? You get in the car and tell it that you want to go to XYZ store, and it takes you to ABC store because it has a sale on. (Or because the ABC store has made a deal with the owner of the car.) I have difficulties in getting search engines to search for what I want, rather than what they think I want.

eric · 27 June 2016

TomS said: I have difficulties in getting search engines to search for what I want, rather than what they think I want.

You've never used Uber I take it? You enter the destination address, not some more vague descriptor like "Safeway" that could be easily misinterpreted. For oddball trips where you don't know the address that might pose some inconvenience, but since our traffic congestion problem is mostly due to people commuting between the office and their homes, 'enter address' for this hypothetical ridesharing auto-cab would work fine for most use cases.

Mike Elzinga · 27 June 2016

eric said:
TomS said: I have difficulties in getting search engines to search for what I want, rather than what they think I want.
You've never used Uber I take it? You enter the destination address, not some more vague descriptor like "Safeway" that could be easily misinterpreted. For oddball trips where you don't know the address that might pose some inconvenience, but since our traffic congestion problem is mostly due to people commuting between the office and their homes, 'enter address' for this hypothetical ridesharing auto-cab would work fine for most use cases.

Driverless Uber cars would at least reduce issues with drivers like Jason Dalton.

Just Bob · 27 June 2016

eric said: Sort of like the Uber app; you push a button on your phone to summon a car, you tell it where you are going, the system matches you to a car and sends it around to pick you up. Both could be incentivised by the use of specail driverless/commute lanes. So I expect the two concepts will go together; driverless cars and fewer cars on the road.

To me, that sounds like fewer cars in existence, but more cars on the road. If I own my own car, it's not going anywhere--off the road--when I'm not commuting. But if riders summon a car when they need one, then the car not only takes them to their destinations, but has to travel, riderless, to pick them up. Ergo, both cars taking people places, plus spooky, ghost-driven cars traveling empty to pick someone up.

eric · 28 June 2016

Just Bob said: To me, that sounds like fewer cars in existence, but more cars on the road...

That might be the case if there's no ridesharing, but as I said to TomS, I think the concept of riderless vehicle is sort of being packaged with the concept of ridesharing. You call a vehicle, so does someone in your neighborhood who is going to roughly the same place, and the same car grabs you both. Which is certainly not going to happen all the time, but for large urban area morning and evening commutes, its a practical certainty. So the concept is not to replace each personal car on the road with a roving riderless vehicle, but to replace three or four personal cars on the road with a roving riderless vehicle. People already do something like this in Washington DC, albeit without the 'driverless' part. This essentially makes that system easier by offering a dependable, door-to-door service to it.

RJ · 28 June 2016

Uber is not any kind of 'sharing' service. It's an unlicensed taxi company that attempts to force municipal governments to withdraw safety and labour regulations without a fight.

I'll never take Uber, and won't deal with any company that relies on it. We already have too many anti-democratic elements with municipal governments in an armbar.

No, not the way forward. Disutilitarian.

By the way, the assertion that massive car reduction is impossible is a self-fulfilling prophecy. We need to reduce carbon emissions in a big way; why not make it a lot easier to travel with shared resources?

harold · 5 July 2016

RJ said: Uber is not any kind of 'sharing' service. It's an unlicensed taxi company that attempts to force municipal governments to withdraw safety and labour regulations without a fight. I'll never take Uber, and won't deal with any company that relies on it. We already have too many anti-democratic elements with municipal governments in an armbar. No, not the way forward. Disutilitarian. By the way, the assertion that massive car reduction is impossible is a self-fulfilling prophecy. We need to reduce carbon emissions in a big way; why not make it a lot easier to travel with shared resources?

As a progressive who cares about labor conditions, emissions, and safety, I will take Uber or Lyft over a taxi service any day. The difference is not that I think Uber and Lyft are great, but that I think the taxi industry is even worse. "Itâs an unlicensed taxi company that attempts to force municipal governments to withdraw safety and labour regulations without a fight." Taxi drivers do unsafe labor for below the minimum wage, and the regulation are there to prevent them from doing what Uber drivers are doing, being able to keep a decent proportion of what the customer pays. The regulations are there to create a monopoly. If the driver is licensed and has no relevant criminal record, and the car is a legal, inspected car, then regulations that actually do serve the public good have been observed. Forcing the driver to pay a lot of money to someone who owns a "medallion" is not serving the public good. There is no social advantage to traditional cab companies. To reiterate, I'm not some libertarian. I believe in strong controls of pollution, strong public transit, decent labor conditions, decent wages, and laws that encourage traffic safety. I just think that traditional cab companies claiming that they promote any of that is a joke.