Hearing aids could soon read lips

Artificial intelligence (AI) could be rolled out on the next generation of hearing aids to help the wearer read lips. Engineers at the University of Glasgow, who developed the technology, say it can even determine what’s being said by people wearing masks.

Data was collected from scanning faces with radar and Wi-Fi signals as they spoke, then used to train an algorithm. It resulted in 95 per cent accuracy in interpreting speech in people without a mask and 83 per cent in those with their lips covered.

Incorporating AI into hearing aids in this way could make it easier for those hard of hearing to understand speech in busy environments. Not being able to take part in conversations in certain settings can be particularly isolating for the elderly.

Dr Qammer Abbasi, of the University of Glasgow’s James Watt School of Engineering and lead author, said: “Around five per cent of the world’s population - about 430 million people - have some kind of hearing impairment.

“Hearing aids have provided transformative benefits for many hearing-impaired people. A new generation of technology which collects a wide spectrum of data to augment and enhance the amplification of sound could be another major step in improving hearing-impaired people’s quality of life.”

At present, hearing aids work by amplifying the ambient noise around a wearer. The success of this approach is limited, however, as it makes it difficult to focus on specific sounds or a single voice amongst a group of people.

The so-called smart hearing aids will be able to overcome this, as they’ll be fitted with a device that collects lip-reading data. Traditional audio amplification will still be used, with the two techniques combined for an all-round better level of understanding.

Researchers in the past have looked into the idea of using cameras to aid lip reading, but this approach raised some concerns about privacy. Face masks, which were common throughout the pandemic, also thwarted the ability for video footage to be captured.

In developing the new technology, scientists collected 3,600 data samples from volunteers repeating vowel sounds. These were then used to teach machine learning and deep learning algorithms to identify the characteristic movements of the mouth.

Since radio frequency signals can easily pass through fabric, face masks didn’t hinder the algorithm’s ability to read vowel formations. When combined with Wi-Fi signals, the technique was even more successful.

Dr Qammer added: “With this research, we have shown that radio frequency signals can be used to accurately read vowel sounds on people’s lips, even when their mouths are covered. While the results of lip reading with radar signals are slightly more accurate, the Wi-Fi signals also demonstrated impressive accuracy.

“Given the ubiquity and affordability of Wi-Fi technologies, the results are highly encouraging, which suggests that this technique has value both as a standalone technology and as a component in future multimodal hearing aids.”