Add LSA-T: "The first continuous Argentinian Sign Language dataset for Sign Language Translation"

[cleong110]

Sign language translation (SLT) is an active field of study that encompasses human-computer interaction, computer vision, natural language processing and machine learning. Progress on this field could lead to higher levels of integration of deaf people. This paper presents, to the best of our knowledge, the first continuous Argentinian Sign Language (LSA) dataset. It contains 14,880 sentence level videos of LSA extracted from the CN Sordos YouTube channel with labels and keypoints annotations for each signer. We also present a method for inferring the active signer, a detailed analysis of the characteristics of the dataset, a visualization tool to explore the dataset and a neural SLT model to serve as baseline for future experiments.

Info:

• Languages: Spanish, Argentinian Sign Language
• Paper: https://arxiv.org/abs/2211.15481
• Dataset URL: https://github.com/midusi/LSA-T
• Samples: 22 hours of sentences.
• Features: video:RGB, text:es, pose:AlphaPose
• Signers:103
• Signs: Not sure. They discuss the Spanish vocabulary, but not the LSA exactly.

They also made their own dataloader in PyTorch.

Poses calculated with AlphaPose: https://github.com/MVIG-SJTU/AlphaPose, "with the Halpe full-body keypoints format" from https://github.com/Fang-Haoshu/Halpe-FullBody, introduced in AlphaPose. Helpfully they also release their models and code at https://github.com/midusi/keypoint-models