abadojack / whatlanggo

Natural language detection library for Go
MIT License
637 stars 64 forks source link

detection problem for short text / training option #19

Open ghost opened 4 years ago

ghost commented 4 years ago

Hi,

Hope you are all well !

I have a problem to detect french language on short sentences like the one below.

Sentence Language Detected Real Language Location
Ras. Esperanto French France
RAS bon. Esperanto French France
PAS DE SOUCI. Portuguese French France
Bien. Spanish French France
RIEN A SIGNALER. Spanish French France
Nickel. Polish French France
Pas assez de recul. Portuguese French France
Je recommande. Dutch French France

Is there a way to train the model with additional patterns/sentences in order to improve detection confidence ?

Btw, I know the location of these sentence, like they are all from France, is there a way to influence the score with an additional parameter like the location ?

Thanks in advance for any insights or solutions !

Cheers, X