Sign language interpreter once without and once with animated mask Copyright: © SignGes Example of a mask superimposed over a sign language video in real time.

Most of us are on social networks and can decide for ourselves whether to link, share, and comment content under our real name or to hide our identity behind a pseudonym. However, for people with a hearing impairment whose native language is a sign language, spoken language is like a foreign language, which is why they also prefer to communicate via sign language videos on the Internet.

Sign languages make use of a variety of different gestures, postures, and facial expressions. Signers thus communicate with their entire bodies, which is why they are also fully recognizable in videos and thus more recognizable to the non-digital public and potentially more vulnerable to attack. Many signers are therefore hesitant to participate in social discussions online.

Dr. Klaudia Grote from the Competence Center for Sign Language and Gestures "SignGes" and Dr. Isaak Lim from the Department of Computer Science 8 at RWTH are researching a way to ensure anonymity for signers in social networks, and thus their participation in digital social life, as well.

Together, the researchers have developed software that places a mask over the sign language videos in real time, hiding the characteristic features of the signers' faces and hands so that they are no longer recognizable as a person.

To do this, a statistical model of a standard face was first created, which was then used to identify distinctive features such as the eyes, eyebrows, and mouth. Next, several standard facial poses were determined. The derived parameters calculate the signer's hand gestures and facial expressions and then translate these into the virtual avatar. The difficulty here is in tracking the fine facial expressions in order to preserve the content completely.

So far, the research team has focused only on translating facial expressions. In the future, however, the software is to be further developed so that their entire body is covered by an avatar, which is essential for the complete anonymization of the signers. Furthermore, it must also be possible to falsify the individual way of signing, since this also allows conclusions to be drawn about the person, comparable to the voice of people communicating by spoken language.