dmlc / gluon-nlp

NLP made easy
https://nlp.gluon.ai/
Apache License 2.0
2.56k stars 538 forks source link

Add usage example for fasttext subword embedding in Pre-trained Word Embeddings tutorial #631

Open szha opened 5 years ago

szha commented 5 years ago

Currently we have the fastText subword embeddings available, but it's not included in the pre-trained word embeddings tutorial which is often the first tutorial people checks out. It would be great to add example usage of fastText subword embeddings and show how it helps representing out of vocabulary words.

The usage of fastText subword embeddings is to simply specify load_ngrams to True when creating gluonnlp.embedding.FastText.

myralucy commented 5 years ago

I'd like to have a try.

szha commented 5 years ago

@myralucy thanks! Here are some pointers that would be useful to this task:

It would be great to have a section in this tutorial to show how to use fastText load_ngrams to handle embedding for unknown words. Feel free to comment here if you need anything.

szha commented 5 years ago

@myralucy did you get a chance to draft the tutorial? Let me know if you need any help!

Aravind-1294 commented 1 year ago

hi siza, is anyone working on this issue.