Project SignON to Develop New Translation App for Sign Language
December 19, 2020
Goal
The SignON project aims to simplify communication between people with hearing loss and those who can hear throughout Europe. Each spoken language has a unique sign language which makes communication extremely complicated. The project is focused on Irish, British, Dutch, Flemish, and Spanish sign languages. In terms of spoken languages, the consortium is concentrating on Irish, English, Dutch, and Spanish.
The application
One of the researchers involved in the project is Dr. Dimitar Shterionov from Tilburg University in the Netherlands where he is the scientific project leader. “The project is very wide-ranging. From a technological research perspective, it is mainly about recognition, analysis, and an understanding of text, speech, and sign language. The app automatically translates into the chosen language using a model that is a hybrid of a deep learning and rule-based model.”
“When something is translated into sign language, it is represented by a 3D avatar. When sign language is translated into a spoken language, it is articulated using artificial speech.” Imagine thereby a computer voice like Siri.
Co-creation
There is a high level of cooperation with deaf and hearing-impaired people during the project. “Communication with the deaf and hard of hearing will steer the development of the app. We will evaluate the app in the interim so that we can incorporate the feedback from this group.” According to Shterionov, this form of co-creation makes the project unique. This approach ensures that the future app will be well-suited to the needs of the user target group.
Partners
Four Dutch partners are involved in the project. Radboud University, the Nederlandse TaalUnie (Dutch Language Association), the Dutch Language Institute, and Tilburg University. A total of 17 European partners are working on the project. The consortium includes various experts in the field of Natural Language Processing (NLP), Machine Learning (ML), Deep Learning (DL), Machine Translation (MT), Linguistics, Deaf Studies, education, and 3D graphics.
Source: Innovation Origins