Watch the unsettling moment a Tesla swerves onto the wrong side of the road after researchers trick its Autopilot AI by placing STICKERS on the ground
- Attack was able to confuse the Autopilot technology on a Tesla Model S sedan
- The car mistook marks on the ground for a lane and veered into oncoming traffic
- Researchers also remotely controlled its windshield wipers and steering wheel
Security researchers have developed a way to trick Tesla’s self-driving technology into driving into the oncoming traffic lane.
All it took was a few inconspicuous stickers on the ground to fool Autopilot on a Model S sedan, before the vehicle’s onboard computers mistakenly steered into the opposite lane, where oncoming traffic was traveling.
The report, published by Chinese tech giant Tencent’s Keen Security Lab, also details adversarial attacks on a Tesla’s automatic windshield wipers and steering wheel.
Researchers have developed a way to trick Tesla’s self-driving technology. By placing stickers on the ground, the Autopilot (pictured) software told the car to drive into the oncoming lane
WHAT IS TESLA’S AUTOPILOT?
Tesla introduced Autopilot in 2014.
Autopilot is the software that powers the car’s self-driving capabilities.
It supports lane centering, self-parking and the ability to automatically change lanes, among other things.
Autopilot relies on cameras, ultrasonic sensors and radar.
These components allow the vehicle to ‘see’ its surroundings, picking up on things like obstacles and traffic lanes.
Data is sent to on-board computers that use machine learning to make judgments on what to do next.
The researchers found they could trick Tesla’s Autopilot technology by placing a series of reflective stickers on the ground.
During demonstrations on a test track, the Model S mistook the stickers on the ground as the actual lane and veered in that direction.
The attack worked on the targeted vehicle even when it was driving in daylight without any interference from snow or rain.
The stickers confused Autopilot’s computer vision, which is what allows the car make decisions in real time.
On-board computers process data from cameras and then use machine learning to decide what to do next, whether that’s avoiding an obstacle or changing lanes.
‘Our experiments proved that this architecture has security risks and reverse lane recognition is one of the necessary functions for autonomous driving in non-closed roads,’ according to the report.
‘In the scene we build, if the vehicle knows that the fake lane is pointing to the reverse lane, it should ignore this fake lane and then it could avoid a traffic accident.’
During demos on a test track, the Model S confused the stickers on the ground as the actual lane and veered in that direction. The attack worked when the car was driving in daylight
While the stickers were small, if the driver pays enough attention to the road, they’d probably be able to spot the marks and right the vehicle into the correct lane.
‘The [researchers] findings are all based on scenarios in which the physical environment around the vehicle is artificially altered to make the Autopilot system behave differently, which is not a realistic concern given that a driver can easily override Autopilot at any time by using the steering wheel or brakes and should always be prepared to do so,’ a Tesla spokesperson told CNET’s Roadshow.
Another section of the researchers’ report details how they were able to gain root access to Autopilot software on a Model S.
In doing so, they could use a game controller to remotely control the car.
They did this by reverse-engineering version 2018.6.1 of Tesla’s Autopilot software to understand some of its underlying technical systems.
Circled in red are the reflective stickers placed on the ground, which tricked Autopilot into swerving into the oncoming traffic lane. Tesla said the attack would be hard to replicate
However, the vulnerability has since been patched in Tesla’s 2018.24 firmware release.
A third and final attack discovered by the researchers allowed them to remotely control the car’s windshield wipers.
They displayed an image generated by an algorithm on a TV screen in front of the car, which tricked the windshield wiper sensors into turning on, falsely believing their was wet weather.
Again, a Tesla spokesperson countered their findings by arguing that the attack on the vehicle’s windshield wipers ‘is not a realistic concern,’ given that the driver has the ability to override Autopilot at any time.