Closed nickchomey closed 1 year ago
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Hey @nickchomey, I apologize for the late reply. The default model (bert-base-cased
) sounds like a good model, but I know it is predominantly trained on English text. As, my work did not require multilingual sentences, so I have little knowledge to answer your question.
If you have any suggestions, please let me know
These might be useful - they use multilingual datasets.
https://huggingface.co/unicamp-dl/mMiniLM-L6-v2-mmarco-v2 https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
The second is a cross-encoder, which is more accurate than biencoder, but very slow. I'm using it as a search re-ranker successfully. It's not clear to me what the first one is. Not sure if a cross encoder can be used with this tool.
Is there a single model that you would recommend using to autocorrect misspellings across common languages? Thanks!