Greek Sign Language Recognition for the SL-ReDu Learning Platform
Katerina Papadimitriou, Gerasimos Potamianos, Galini Sapountzaki, Theodore Goulas, Eleni Efthimiou, Stavroula-Evita Fotinea and Petros Maragos
Seventh International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual (SLTAT 2022)
Marseille, France, 24 June 2022
Abstract for Book of Abstracts
There has been increasing interest lately
in developing education tools for sign language (SL) learning
that enable self-assessment and objective evaluation
of learners' SL productions,
assisting both students and their instructors.
Crucially, such tools require the automatic recognition of SL videos,
while operating in a signer-independent fashion
and under realistic recording conditions.
Here, we present an early version of a Greek Sign Language (GSL) recognizer
that satisfies the above requirements,
and integrate it within the SL-ReDu learning platform
that constitutes a first in GSL with recognition functionality.
We develop the recognition module
incorporating state-of-the-art deep-learning based visual detection,
feature extraction, and classification,
designing it to accommodate a medium-size vocabulary of isolated signs
and continuously fingerspelled letter sequences.
We train the module on a specifically recorded GSL corpus of multiple signers
by a web-cam in non-studio conditions,
and conduct both multi-signer and signer-independent recognition experiments,
reporting high accuracies.
Finally, we let student users evaluate the learning platform
during GSL production exercises,
reporting very satisfactory objective and subjective assessments
based on recognition performance and collected questionnaires, respectively.
Conference Manager (V2.61.0 - Rev. 6487)