Model for working with multi-languages

[danabez1]

Hello, I am working with data that contains a mix of languages (e.g. English, France, Spanish) I would like to use pre-trained sentence-bert model to find similarities between pairs of text. In some cases, text1-text2 are from the same language, and in others a mix of 2 languages.

• Which model do you recommend me to the work with?
• Is 'all-mpnet-base-v2' suitable for my case? Is the prefix 'all' implies it is suitable for multi-languages as well?
• What is the advantage of using multilingual models, like 'paraphrase-multilingual-mpnet-base-v2'? Thank you

[nreimers]

The all-* models are only trained on English. The all means 'all-training-datasets'.

You must use one of the multilingual models like paraphrase-multilingual-mpnet-base-v2

[danabez1]

Thank you :)

[Tortoise17]

@nreimers I have one question. as example. If we have text as dataframes in English and query is in German or French (es example) , will it return the answer match in German or it is not possible with such kind of problem with multi-language model.

[nreimers]

Languages are aligned, so it will find the closest match irrespective of the language

[Hayat21]

Hello there I hope your are doing well Can you tell me which model support arabic language, and must be fast and performant for semantic Thanks

[jaideep11061982]

@nreimers @danabez1 how many languages does paraphrase_ml supports ? does it supports 100 languages or 15 languages ?