Researchers have developed technology that monitors a driver's movements and warns them when they are about to make a mistake.

No one has come up with a foolproof system for autonomous driving but driver assistance programs that help motorists control their vehicles are becoming more sophisticated all the time.

Cornell and Stanford University researchers have developed a system that anticipates what a driver is going to do a few seconds before it happens using cameras and a new computer algorithm.

Some cars are already equipped with safety systems that monitor a car’s movement and warn if there is an unsafe turn or lane change, the researchers noted. But the warning comes too late, after the driver has acted.

However, by observing the driver’s body language and considering that in the context of what’s happening outside the car, a computer algorithm computes a probability that the driver will turn, change lanes or continue straight ahead, according to the researchers from the two universities.

“There are many systems now that monitor what’s going on outside the car,” explained Ashutosh Saxena, assistant professor of computer science at Cornell. “Internal monitoring of the driver will be the next leap forward.”

Saxena and Ashesh Jain, a Cornell graduate student, will describe their system in a workshop on “Model Learning for Human-Robot Communication” at the 2015 Robotic Science and Systems conference July 16 in Rome.

By combining driver anticipation with radar or cameras to locate other vehicles, the car’s safety system could warn the driver when the anticipated action could be dangerous. The warning might be a light, a sound or even a vibration. “If there’s danger on the left, the left side of the steering wheel or the seat could vibrate,” Jain suggested.

(Most drivers still listen to AM/FM radio. For more, Click Here.)

The system also might give an “illegal turn” message if the driver was planning, say, to turn the wrong way on a one-way street by drawing on street maps and GPS information.

Saxena and colleagues recorded video of 10 drivers, along with video of the road ahead, for 1,180 miles of freeway and city driving over a period of two months. A computer using face detection and tracking software identified head movements and learned to associate them with turns and lane changes, so that the final system can anticipate possible actions the driver may take. The computer continuously reports its anticipations to the car’s central safety system.

In a test against another data set of videos with different drivers the system correctly predicted the driver’s actions 77.4% of the time, anticipating an average 3.53 seconds in advance. Those few extra seconds might save lives, Saxena said.

(Click Here for details about why majority of car buyers are ready to buy online.)

The system still needs refinement, the researchers noted. Six percent of the time, they found, face tracking was confused by shadows of passing trees and other lighting variations. The system also can be misled by drivers interacting with passengers. In some situations, such as turning from a turn-only lane, drivers don’t always give the same head cues.

Sometimes they rely on short-term memory of traffic conditions and don’t turn their heads to check. It may come down to tracking eye movements, the researchers said.

Jain said incorporating it in a complete safety system is a job for automakers. Future improvements may include infrared cameras to observe at night and 3-D cameras for greater accuracy. Other inputs may be added, such as tactile sensors to monitor pressure on the steering wheel, and cameras or pressure sensors to observe what the driver’s feet are doing – perhaps to anticipate braking.

(To see the ten cities with the greenest car shoppers, Click Here.)

Observations could be extended to other activities, such as whether the driver is looking at a phone or a watch. The system could link directly to such wearable technologies, Jain said.