Cameras that translate mouth gestures into words!

Toronto, Aug 29 ( People with speech disability will now be able to speak, literally.

A Canadian professor at the University of Toronto says heat-seeking infrared camera can pick up mouth gestures of a person with speech disability and then pick the letters of the alphabet to translate the gestures into verbal message.

According to Prof Tom Chau, who is senior scientist at Bloorview Research Institute of the university, infrared thermography can help people with speech disabilities to interact and perhaps eventually communicate.

Infrared thermography, he says, refers to the measurement of radiation or heat emitted by the surface of an object, including the mouth.

Infrared cameras with specialized lenses can pick up this radiation or heat and focus it on a plane of infrared detectors.

Explaining his breakthrough, Chau said his research involves exploiting the temperature changes associated with the opening and closing of the mouth.

“The human body is an emitter of radiation (or heat) and the radiation that is emitted can be measured,” he was quoted as saying in a university statement.

He said, “In the (human) face there is a complex network of blood vessels. When you experience different emotions there is different flow of blood through the face and this causes temperature changes we measure non-invasively using a thermal camera.”

Chau, who came up with this idea while working with a patient with speech disability, said, “One thing (this patient) could do reliably was open and close his mouth and that is a very tricky thing to capture.

“We came up with the idea that a person is warmer when they open their mouth. As long as there is heat there, the camera can pick it up. We are looking for a temperature change and motion when a person opens and closes his mouth.

“The mouth opening and closing just acts as a switch. That switch can be used to select letters of the alphabet from an on-screen keyboard or images in a picture communication board.

“That is how someone can use the one action (mouth opening/closing) to make choices. Now this individual is able to type on a computer using the thermal camera switch.”

In the lab, he said, the camera is pointed at the patient while he sits in front of the computer and every time he indicates a letter he wants, he opens his mouth and the camera captures it.

“A couple of weeks ago, the individual said his first word in his life, in his 26 years. This was in the lab and his mother was there. He was typing letter by letter (using his mouth as a switch). He typed m-u-t-h-e-r. His mom realized he was saying mother and she just broke down in tears. It was a dramatic moment. It has become such a liberating technology for this individual.”

The university statement says Chau and his team are the first in the world to use infrared thermography for this purpose.