sign-language-processing / spoken-to-signed-translation

a text-to-gloss-to-pose-to-video pipeline for spoken to signed language translation
https://sign.mt/?sil=sgg&spl=de
MIT License
43 stars 11 forks source link

English to ASL #25

Open MuhammadAasharibNawshad opened 8 months ago

MuhammadAasharibNawshad commented 8 months ago

Hello, This repository does not have English to ASL. But https://sign.mt has english to ASL working. Can you please guide how to add English to ASL translation?

bipinkrish commented 8 months ago

Hello, This repository does not have English to ASL. But sign.mt has english to ASL working. Can you please guide how to add English to ASL translation?

I have the same doubt

AmitMY commented 8 months ago

This repository allows loading any lexicon. For example, we load the suisse lexicon, but any lexicon can be added. sign.mt is an independent website, that uses this project, but with other lexicons. Adding ASL to this repository will be as simple as the suisse lexicon, all there needs to be is:

see our example lexicon.

bipinkrish commented 8 months ago

sign.mt is an independent website, that uses this project, but with other lexicons.

Can you please point me where in sign/translate you used lexicons or the models you trained from it?

All I found was here an I am not sure how to use those in python even if I re-trained using this

AmitMY commented 8 months ago

In sign/translate, the backend for the spoken-text-to-pose is closed source for multiple reasons. Ideally, everything will work client side only, but this is not yet the case.

bipinkrish commented 8 months ago

In sign/translate, the backend for the spoken-text-to-pose is closed source for multiple reasons.

Is it because of licensing ?

AmitMY commented 8 months ago

Among other things, yes.

MuhammadAasharibNawshad commented 8 months ago

Thankyou for clarifying. One thing i want to know is that whether this repository be useful for text to gloss if I want to convert english text to ASL gloss and then run GLoss lookup for custom ASL glosses?

If yes then how? as currently it looks like it supports other language but not english. If No, Then how can i create glosses from text for english for ASL?

AmitMY commented 8 months ago

Using the spacylemma module or simple module, you can lemmatize English text into pseudo-glosses. If you'd like the rules module to work for ASL, you would have to implement the rules of the grammar.

MuhammadAasharibNawshad commented 8 months ago

@AmitMY Thankyou for clarifying. I understand the pseudo-glosses part. But can you please specify resources/logic for english ASL glosses for grammar rule? is it implemented somewhere or if not then any description about the rules which can help in implementation?

AmitMY commented 8 months ago

In my opinion, you should start from the spacylemma implementation, and only when everything works for you consider adding grammar manipulations (rules)

There are many resources on grammar