Good drivers can predict how pedestrians will move, now AI is helping self-driving cars do the same

You pull up to a busy intersection in your car. You wait for the light to turn green and scan the crossing. Just as you think it’s safe to go, a fast-moving cyclist flies across the road, narrowly missing your car. “Phew,” you think, “that was close!”

As a driver, predicting how pedestrians and cyclists will behave at crossings can be a real challenge. Your brain is good at picking up on clues such as people’s gait, speed or head position to help inform your predictions, but when you add technology into the mix, things start to get really interesting! Thanks to smartphones, pedestrians are walking around our cities in an increasing state of distraction. Dubbed ‘Smartphone Zombies’, many pedestrians are so engrossed by the glow of their phone’s screen that they are almost oblivious to where they are going.

Human drivers are good at recognizing this type of behavior and predicting where people may move in a few moments. But autonomous cars struggle to make the same sort of judgment calls. Fortunately, researchers are training artificial intelligence (AI) systems to become better at predicting pedestrian movement. Here’s how autonomous cars are using data to make better decisions.

Self-driving cars Vs Smartphone Zombies
Autonomous cars are already here, and they navigate our roads thanks to perception systems such as cameras and LiDARs that map their surroundings. But the rise of ‘smartphone zombies’ presents a problem; just perceiving the world isn’t enough. Good human drivers can predict where a person or car will move in a few moments. Self-driving cars need to learn how to do the same. One of the greatest challenges facing the development of self-driving cars is making them better at predicting where pedestrians and other cars will move.

The ‘Intersection Dance’
According to Matthew Johnson-Roberson, an associate professor of engineering at the University of Michigan, the challenge stems from the fact that traffic lights don’t tell cars and pedestrians when it's safe to go. “It’s just a really hard social situation,” he says, almost “like a dance”. Pedestrians need to be looking up, trying to make eye contact with drivers, but instead, they are usually glued to their smartphone screens. Obviously, there are some constraints, or ground rules, that govern human behavior – people need to be balanced when they walk and they can’t fly. Nevertheless, AI systems struggle to predict human movements and actions.

Developing an AI Neural Network
To help train AI systems to predict pedestrian movement, researchers at the University of Michigan led by Johnson-Roberson gathered real-world data by parking autonomous cars near intersections in Ann Arbor, Michigan. Their job? To watch and observe. This research helped the team to train an AI system called a ‘neural network’ that can accurately predict pedestrian motion. According to a new study, this system is surprisingly accurate with people who are up to 148 feet away. In almost all cases, it can predict where a person will move “within a body length of where the person actually was,” says Johnson-Roberson.

How it works
The AI system is known as a ‘biomechanically inspired recurrent neural network’. It can interpret pedestrian gait to predict their future position and pose in three dimensions. The researchers used a vehicle with four Velodyne LiDARs and two stereo camera pairs in combination with ‘PedX data’ about the pedestrians in the intersection. To navigate safely, the system needs to know two factors:

  • The pedestrian's pose
  • Their global location

The study allowed the team to observe how pedestrian's body movements can alter, depending on their actions. Heavy bags causes people to lean to one side and compensate, for example. The system could even be trained to recognize when people are using smartphones, due to the orientation or their heads.

Saved from being smooshed
The AI Neural Network developed by Johnson-Roberson’s team could potentially be used to help autonomous vehicles navigate the roads around fragile humans. The aim is to develop the system to the point where its prediction capabilities have human-like levels of accuracy. Even if people aren’t looking around them when they cross the road, self-driving cars will help ensure that they live to cross another day.