Ethics Homework

Topic: Tesla driver dies in first fatal car crash

Wenhan Xia, Julia Currie, Divya Gupta, Nicholas Sarkis, Norman Chen, Eric Cole

On May 7, 2016, Joshua Brown was driving his 2015 Tesla Model S from Cedar Key to Ocala, Florida. The trip lasted 41 minutes, 37 of which the driver relied upon the vehicle’s internal driving assists, principally Traffic-Aware Cruise Control (TACC) and Autosteer lane-keeping, despite a combined two minutes of visual and auditory warnings to the driver from the vehicle to provide manual steering input. At 4:36 PM, the Tesla struck a semitrailer that had turned in front of the car’s path, resulting in the car travelling off the highway and striking a utility pole. Brown was killed in the event (NTSB 2016).

Joshua Brown’s death is contemporaneous with the widespread integration of automation into common consumer products and services, from “self-driving cars” to automated restaurants (Washington Post). Due to the pace of its advancement, however, automation is proving difficult to regulate. According to the National Transportation Safety, vehicular automation in particular has no enforced regulatory guidelines, only a set of recommendations set forth by the National Highway Traffic Safety Administration (NHTSA): “[vehicle] manufacturers are not required to follow the vehicle performance guidance in the [Automated Vehicle] Policy, but it provides a framework for operation, testing, and safety assessment of vehicle automation” (NTSB p 32). The lack of legally enforced Automated Vehicle (AV) policies and a general lack of transparency regarding how AVs make ethically divisive decisions, including whether to preserve the driver’s life over the expense of others, raises the question of whether consumers should blindly trust AV technologies. Owing to the number of stakeholders consciously or unconsciously invested in AVs – AV consumers and manufacturers have direct stakes in the AV market, though all people living in places where AV technology is distributed are affected by safety policies – NHTSA must dissuade consumers from blindly trusting automated technology by requiring consumers to read about and fully understand AVs’ technical limitations, and requiring manufactures to fully disclose AV decisions during ethically divisive situations. These conclusions are drawn from justice tests. Virtue tests would be less appropriate in this context, since they would directly address the reputation of a body of stakeholders, such as Tesla, Inc., while relegating the safety of the public and AV consumers to secondary importance.

To determine the fairest future course of action regarding AV policy making, Tesla Inc.’s AV technical documentation and technology must first be addressed. According to NTSB’s incident report detailing Joshua Brown’s crash, the Brown’s Tesla Model S falls under a Society of Automotive Engineers (SAE) automation ranking of 2, which indicates that the car was only partially automated, and that “human driver[s] [must] monitor the driving environment” (NTSB p 24). Though this ranking is not explicitly mentioned in the Tesla Model S manual, the manual does make numerous notices that detail the company’s idea of proper AV usage, including “Warning: Traffic-Aware Cruise Control is designed for your driving comfort and convenience and is not a collision warning or avoidance system. It is your responsibility to stay alert, drive safely, and be in control of the vehicle at all times” (Tesla p 67). The manual also lists situations in which the Tesla’s AV technology may fail, such as when

The manual clearly states the limitations of the car’s technology, and provides multiple disclaimers regarding AV misuse. Whether Joshua Brown read these limitations and disclaimers, however, is unknown, since reading car manuals is typically performed at the car owner’s discretion. In an video called “Tesla Autopilot v7.0 Intro Video”, Brown discusses a firmware update to the Model S. Toward the end of the video, he discusses a number of the AV misuse safeguards, including auditory and visual warnings, and an automated shutdown mechanism that activates when a driver fails to heed the preceding warnings. Despite his nominal understanding of AV limitations, there is evidence that Brown grossly overestimated his car’s ability to replace human judgment. In a YouTube video titled “Autopilot Saves Model S”, Brown captures his Model S avoiding a sideswipe collision while in autopilot mode. Brown captions the video with a lengthy discussion about his impressions of the car’s ability, stating:

“[The Tesla] did great. I have done a lot of testing with the sensors in the car and the software capabilities. I have always been impressed with the car, but I had not tested the car’s side collision avoidance. I am VERY impressed. Excellent job Elon [Musk]! Note: I have over 39,000 miles on the car and I’ve had it since mid-July 2015. Hands down the best car I have ever owned and use it to its full extent.It has done many, many amazing things, but this was one of the more interesting things caught on the dashcam (Autopilot 2015).”

The car’s lucky maneuver likely gave Brown the false impression that the car could reliably make life-determining decisions without human intervention. Furthermore, Brown makes an unsubstantiated claim about vehicular learning in the comments section:

“I’m a technical guy and I am truly amazed at how quickly it is learning. It has gotten MUCH better at curvy roads. It slows down to an appropriate speed for most curves (though it doesn’t seem to get ALL of them right right yet). It also does NOT dive off exits any more. That used to be a 50/50 change on whether it would follow the lane marking off an exit if you were driving down the right lane.”

There exists no information on the Tesla website about vehicular learning (Full Self Driving). This false impression underscores the need for laws enforcing car manual reading and comprehension assessments prior to vehicular purchase. The need for compulsive manual reading and comprehension assessments will become even more critical in the near future, as the degree of vehicular automation increases. Suppose, for example, that a group of schoolchildren is standing in front of an AV’s path, and that the only way to save the children is to hit a brick wall and kill the driver. Drivers must know in advance how their vehicles will react to divisive situations.

A justice test concerning AV consumers, AV manufacturers, agencies such as NHTSA, and the general public as stakeholders reveals the need for compulsive presale AV manual reading and comprehension tests, and full disclosure of how AVs handle ethically divisive situations. Presale manual reading and comprehension assessments at manufacturer dealerships would benefit the public and AV drivers, since the drivers would, in principle, know how to appropriately use AV technology and understand the limitations of said technology, thus decreasing the probability of an AV-related road accident. No presale manual reading or comprehension assessments might be convenient for the consumer, but would increase driver negligence and thus increase the risk of an accident. Furthermore, consumers may argue that compulsory comprehension assessments in the form of a presale exam, for example, would be demonstrative of government agencies exerting excessive social control and limiting consumer freedoms. Though unprecedented for automobile sales, a comprehension exam would eliminate false ideas about AV abilities, such as Joshua Brown’s impression that his vehicle was “learning.” The need for awareness of AV technical limitations carries greater weight than consumers’ wariness regarding increased governmental control of automobile sales.

Regarding disclosure of how AVs handle ethically divisive situations, AV manufacturers may argue that such disclosure would damage sales, since the manufactures’ ethical philosophies–utilitarianism (maximize the number of lives saved) and liberalism (save the driver’s life at all costs) being two such examples–may contradict customers’ philosophies. In a paper titled “The social dilemma of autonomous vehicles”, authors Jean-François Bonnefon, et. al., state that

“even though participants approve of autonomous vehicles that might sacrifice passengers to save others, respondents would prefer not to ride in such vehicles…. Respondents would also not approve regulations mandating self-sacrifice, and such regulations would make them less willing to buy an autonomous vehicle (Science, 2016).”

It would seem, then, that even if regulatory bodies enforced utilitarianism, to which prospective AV consumers responded negatively, fewer people would purchase AVs, thus damaging manufacturer profits. Regardless, safety takes precedence over profitability. Drivers must have the right to know how their vehicle will behave in life-determining situations, and the general public, when confronted with AVs, must be able to anticipate how such vehicles will behave. A “tunable philosophy” option on AVs, however, may serve the interests of manufacturers, drivers, and the general public. Manufacturers would surrender ethical control to consumers, thereby bypassing profit losses and loss of consumer agency. The general public could be alerted of an AV’s “ethics setting” via an external indicator, such as a lighting or coloring scheme unique to each setting. An ethical dilemma occurs, however, when considering passengers, since not all passengers may agree on an ethics setting. If ethical control is given to the driver, passengers must understand and accept the driver’s wishes prior to departure.

Automation is powerful and pervasive, and may make life safer and more convenient if properly implemented. When automated processes directly affect human safety, stringent policies must be created, and manufacturers must be held accountable for upholding these policies. Given the number of automobile drivers, the proper use of automated vehicles and full disclosure of their behaviors in ethically divisive situations is particularly important, and regulatory bodies such as NHTSA are responsible for ensuring the accountability of AV manufacturers, and the safety of AV drivers and the general public. Preliminary policies such as legally enforced vehicular manual reading, comprehension assessments, and thorough disclaimers will ease the transition of automated vehicles from a nascent to a mature and safe mode of transportation.

References