|
KoSign Sign Language Translation Project: Introducing The NIASL2021 Dataset
Mathew John Huerta-Enochian, Du Hui Lee, Hye Jin Myung, Kang Suk Byun and Jun Woo Lee
Seventh International Workshop on Sign Language Translation and Avatar Technology: The Junction of the Visual and the Textual (SLTAT 2022)Marseille, France, 24 June 2022
Abstract for Book of Abstracts
We introduce a new sign language production (SLP) and sign language translation (SLT) dataset, NIASL2021, consisting of 201,026 Korean-KSL data pairs. KSL translations of Korean source texts are represented in three formats: video recordings, keypoint position data, and time-aligned gloss annotations for each hand (using a 7,989 sign vocabulary) and for eight different non-manual signals (NMS). We evaluated our sign language elicitation methodology and found that text-based prompting had a negative effect on translation quality in terms of naturalness and comprehension. We recommend distilling text into a visual medium before translating into sign language or adding a prompt-blind review step to text-based translation methodologies.
START Conference Manager (V2.61.0 - Rev. 6487)