Microsoft Research: The Kinect Sign Language Translator

How can we use Kinect for sign language translation? Microsoft Research in China has created Kinect Sign Language Translator, a real-time machine translation of sign language using Kinect which understands Chinese sign language and translates it into Chinese in real time using Kinect's sensor set--and vice versa. 

With the aim of easing the communication for sign language speakers in commercial and employment settings, the Kinect captures the gestures, while machine learning and pattern recognition programming help interpret the meaning. The system is capable of capturing a given conversation from both sides: a deaf person who is showing signs and a person who is speaking. Visual signs are converted to written and spoken translation rendered in real-time while spoken words are turned into accurate visual signs.

It is currently in prototype phase, taking five people to establish the recognition patterns for just one word. Only 300 Chinese sign language words have been added out of a total of 4,000 so far, but the implications of the technology to help people are enormous. 

We also recommend watching: "I Want To Say: Touch Screens Give Autistic Kids a Voice" and "Microsoft's 'Kinect Accelerators': Turning Hacks into Startups ".

Microsoft Research: The Kinect Sign Language Translator

Microsoft Research (MSR) is the research division of Microsoft created in 1991 for developing various computer science ideas and integrating them into Microsoft products. It employs Turing Award winners and many other recognized experts in computer science, physics, and mathematics.