STUDENT PROJECT SHOWCASE: GERONTECHNOLOGY
Project Name: SignChat - Realtime Sign Language Translator
- Discipline involved: IT
Real-time sign language translation application
A group of HD in Software Engineering students from IVE(Sha Tin) developed a prototype of a sign language interpretation system as their Final Year Project in AY2019/20. Currently, another group of students are working on the enhancement and extension of the project.
The prototype comes with a 3D hand tracking system and sign language interpretation AI using Neural Network. Through capturing 3D coordinates of hand feature points using a smartphone camera, the AI interprets the meanings of hand signs from a pre-trained AI model.
There is a population of more than 150,000 with hearing impairment in Hong Kong. Sign Language is a useful means to communicate in the deaf community. However, sign language is not popular among the hearing community, and there is also a limited number of sign language interpreters. This forms a communication obstacle between hearing people and deaf people. The project aims to help the deaf and dumb express themselves using sign language to people who do not have relevant training in Hong Kong.
At this stage, an iOS application has been developed alongside a cloud-based sign language recognition system. Besides, a capture app has been developed for new hand sign training. Currently, the system is able to process hand-sign sequences of 5 to 6 seconds (about 1-2 sentences) and recognize around 20 hand signs.