search

sign / translate

Effortless Real-Time Sign Language Translation
https://sign.mt
Other
414 stars 74 forks source link

Model for Sign Language Detection #122

Closed priyanshmathur closed 7 months ago

priyanshmathur commented 7 months ago

Description

Hi @AmitMY. I wanted to know which model you are using for webcam translation of sign language gestures into words.

I would like to know the complete pipeline of the same if possible. I understand your project employs a bi-directional way of communication, but I simply wanted the model which understands Indian Sign Language gestures via webcam and display the tokens or predicted words.

Thank You for your help.

AmitMY commented 7 months ago

Here we have a demo of sign language segmentation + recognition - It is backend only, so doesn't work directly from the webcam. What you are looking for is "Sign Language Recognition", not "Detection" (see definitions here https://research.sign.mt/#tasks)

In this specific project, while I welcome contributors adding sign language recognition (video->gloss) my main research direction is sign language transcription (video->SignWriting) and so this is not my focus.

Feel free to contribute a model or help develop the code to support this pipeline as well.