It’s not that long ago that self-driving cars (also known as autonomous vehicles or driverless cars) seemed like the domain of science fiction. Yet in reality, researchers have been conducting experiments on them since the 1920s.  

Now, we’re heading into an exciting future where we may no longer be drivers but passengers. Imagine sending your kids to soccer practice, instead of having to drive them – safe in the knowledge that the car will get them there all by itself. Or picture a world where no one owns a car at all anymore, but simply calls one when needed.

Autonomous driving opens up an array of possibilities – but there are still some key challenges to overcome to make truly autonomous vehicles a reality.

In order for a car to ‘see’, ‘hear’, ‘understand’, ‘talk’, and ‘think’, it needs a huge array of video, image, audio, text, LIDAR and other sensor data to be correctly collected, structured and understood by Artificial Intelligence (AI) or machine learning models.

Most of these models are based on deep learning networks, often called neural networks – complex mathematical systems that can learn tasks by analyzing vast amounts of data. By processing thousands of cat photos, for instance, a neural network can learn to recognize a cat.[1]

You may never have heard of neural networks, but in fact, you’re likely to interact with them frequently. This is the same technology that recognizes faces in the photos you post to Facebook, identifies commands to your cellphone, and translates between languages on internet services like Microsoft Skype.

It’s this technology that’s hastening the advance of autonomous robots, including self-driving cars. But it faces significant challenges – the biggest one is to process the amount of data generated from sensors and derive the right decisions based on that data. Not to mention the large amount of data required for these networks to learn before they are even deployed.

 “If you show a child a picture of a cat, they will recognize a real cat without thinking,” says Dr David Mueller, our Lead Strategist in Electronics. “But you need to show an AI system millions of pictures of cats in order for it to then reliably identify another cat. And this is the same problem you have in self-driving cars. Just think about the amount of work your brain is doing when you drive – there are so many decisions that you’re making without even thinking about it. For example, if another car suddenly pulls out in front of you, your foot will be on the brake – probably before you’ve even realized you’ve done it. You’ve seen something happen, processed it, understood its implications, and taken action – all in moments and just based on your limited
previous experience.”

For a self-driving car to learn to do the same thing requires enormous amounts of input data and very clever algorithms.

“The brain has incredibly efficient and fast heuristics or ‘rules of thumb’ to make decisions,” continues Mueller. “This is true even in situations where many pieces of information are missing. Our mostly subconscious intuition does a phenomenal job, particularly in situations not experienced before.”

It’s vital to think about how to teach a driverless car to make adjustments it may not have been trained to make – for example, when a situation it hasn’t encountered arises.

Just one autonomous vehicle could generate up to a staggering 5TB per hour. That's the equivalent of 1,250 movies or 2,500 hours of HD video.[3] This massive amount of data has to be processed in real-time. 

The big challenge here is how to efficiently manage all the data that’s generated during these trips.

“The computers running the most recent self-driving cars are effectively small supercomputers,” explains Mueller. “Nvidias’ latest development, which aims to achieve Level 5 autonomous driving in 2022, delivers an impressive 200TOPS (trillions of operations per second) using 750W of power.”[4]

But spending 750W an hour on processing will have a noticeable impact on the driving range of electric vehicles.

Mueller suggests that, “there may be more elegant and energy efficient ways to create self-driving cars by using neuromorphic chips.”

Neuromorphic chips are designed to imitate the network of neurons in the brain. Rather than just processing 1s and 0s, these so-called spiking neural networks (SNN) operate by trading tiny bursts of electrical signals, ‘firing’ or ‘spiking’ only when input signals reach critical thresholds, as biological neurons do.

Combined with the right sensors these SNN operate extremely efficiently, because they only start processing if something meaningful has changed.

They also perform better because they can simultaneously store and process information, just like the neurons and synapses in the human brain do. While conventional computers run commands sequentially, constantly moving data packets received back and forth from the memory to the processor, neuromorphic computers process and store data largely at the same time. This makes them both faster and far more energy efficient, just like the human brain.  

The hope for such chips is that, by operating a bit more like the brain, they can help AI systems like autonomous vehicles learn skills and execute tasks more efficiently.

“Today truly neuromorphic chips are still in an early stage, particularly those employing spiking neural networks. But the potential they have to enable autonomous driving is striking,” says Mueller.

Every digital advance in recent years, from smartphones to 5G networks, relies on advanced semiconductor materials. And self-driving cars are no exception. To create the advanced computing required for these systems, equally advanced, high-quality semiconductor materials and manufacturing are needed.

Because of this, consultancy firm PwC estimates that sales of semiconductors to the automotive industry will increase by 11.9 percent by 2022.[5] This trend is closely related to the huge market potential of autonomous cars, as well as the continuing electrification of vehicles and the development of onboard assistants and in-car entertainment systems.

“Our high-tech materials are enabling the development of all the increasingly powerful microchips, sensors and other technology that form the backbone of self-driving vehicles,” says Kai Beckmann, our CEO of Electronics. “Everything from the onboard computing to the GPS and display screens relies on efficient, advanced semiconductor solutions.”

There’s a lot of disagreement among experts on whether genuinely autonomous driving will be a reality in the near future. (See our infographic for autonomous driving classifications.)[1]   

Consulting firm McKinsey predicts that “once technological and regulatory issues have been resolved, up to 15 percent of new cars sold in 2030 could be fully autonomous.” They also suggest that fully autonomous vehicles are likely to be commercially available towards the end of 2020.[6]

However, other commentators are considerably more cautious. An article published in Nature Machine Intelligence in 2019 states that: “If we are to realize the potential of self-driving cars, we need to recognize the limits of machine learning. We should not pretend self-driving cars are around the corner: it will still take substantial time and effort to integrate the technology safely and fairly into our societies.”[7]

What is clear is that making autonomous driving a reality isn’t simply a case of creating the right technology – although that is, of course, a big part of it. External systems, public policy, and consumer sentiment will need to change alongside the technology to enable a future where the cars drive themselves.

_{[1] https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
[2] https://datacenterfrontier.com/rolling-zettabytes-quantifying-the-data-impact-of-connected-cars/
[3] https://experience.dropbox.com/resources/how-much-is-1tb
[4] https://www.counterpointresearch.com/new-entrants-autonomous-vehicle-solutions-will-drive-market-coming-decade/
[5] https://www.pwc.de/de/technologie-medien-und-telekommunikation/opportunities-for-the-global-semicondur-market.pdf
[6] https://www.mckinsey.com/industries/automotive-and-assembly/our-insights/disruptive-trends-that-will-transform-the-auto-industry
[7] https://www.nature.com/articles/s42256-019-0046-z}