facebookresearch / fastText

Library for fast text representation and classification.
https://fasttext.cc/
MIT License
25.76k stars 4.71k forks source link

So many wrong predictions in latest version? #1326

Open siara-cc opened 1 year ago

siara-cc commented 1 year ago

Hi, I am using fasttext in Python installed from this repo and Java installed from here: https://github.com/vinhkhuc/JFastText

JFastText seems based on 0.8.22 of fastText. I am getting differences in predictions, that too the older version seems correct.

For example, when using the latest version using Python:

import fasttext
fmodel = fasttext.load_model("lid.176.bin")
fmodel.predict('Half...')
fmodel.predict('LOVE')
fmodel.predict('batman forever')
fmodel.predict('Very swollen')

Output:

(('__label__pt',), array([0.31706801]))
(('__label__ja',), array([0.61443537]))
(('__label__ru',), array([0.15859173]))
(('__label__de',), array([0.45038238]))

which are incorrect, since these are clearly english words.

I am not sure why this might be happening, but It would seem that the latest version 0.9.2 seems to be giving incorrect predictions, while earlier version 0.8.22 was giving correct results.