Text classification multilanguage

dotnet / machinelearning-modelbuilder

Simple UI tool to build custom machine learning models.

Creative Commons Attribution 4.0 International

267 stars 56 forks source link

Text classification multilanguage #2636

Open IzzyHibbert opened 1 year ago

IzzyHibbert commented 1 year ago

ML.NET 2.0 comes with Text classification. The Text classification scenario is based on NAS-BERT, which is trained on English Wikipedia plus BookCorpus.

When using text classification for a not English language, the accuracy is not as good as expected. Is there a way to solve this, even by fine- tune ?

LittleLittleCloud commented 1 year ago

@michaelgsharp Does ML.Net text classification support other language? If it does we can expose it in model builder.

Otherwise what step should we take for adding support for another language

IzzyHibbert commented 1 year ago

Any chance for some light here ? Thanks a lot. @michaelgsharp + @LittleLittleCloud