Computer Learns Sign Language by Watching TV
From: New Scientist - 07/08/2009
By: Colin Barras

Software developed by researchers at the University of Oxford and the
University of Leeds has autonomously determined the basics of sign language
by watching TV programs that are subtitled and signed. The researchers first
designed an algorithm to recognize gestures, without assigning a definition
to those gestures, made by a signer on TV. The software focuses on the arms
to determine the rough location of the hands, and identifies flesh-colored
pixels in those areas to identify precise hand shapes. The researchers
exposed the system to about 10 hours of TV footage that contained both sign
language and subtitles, and tasked the software to learn the signs for a mix
of 210 nouns and adjectives that would appear several times throughout the
footage. The program analyzes the signs that accompany each of the words
whenever they appear in the subtitles. When it was not obvious which part of
a signing sequence corresponded to the given word, the system compared
multiple occurrences of the word to isolate and identify the correct sign.
The software correctly learned 136 of the 210 words. University of Leeds
researcher Mark Everingham says some of words have different signs, so a 65
percent success rate is quite high given the complexity of the task.
Researchers at the University of Surrey have developed a similar system that
scans all of the signs in a video sequence to identify signs that appear
frequently and likely represent common words. Both approaches could be used
to create a way to automatically animate digital avatars capable of signing
fluently for deaf TV program viewers. 

Read the entire article at:
http://www.newscientist.com/article/dn17431-computer-learns-sign-language-by-watching-tv.html

Video:
Software can learn sign language by watching TV
http://www.newscientist.com/articlevideo/dn17431/28820633001-computer-learns-sign-language-by-watching-tv.html

Link:
Mark Everingham
http://www.comp.leeds.ac.uk/me/