search

mwydmuch / extremeText

Library for fast text representation and extreme classification.
Other
150 stars 16 forks source link

Trying to reproduce Fasttext model but get Runtime Error while testing #6

Closed Umar-Ayub closed 5 years ago

Umar-Ayub commented 5 years ago

Hi I am trying to reproduce a fasttext model using extremetext in python(I just downloaded extremetext and could use some of its functionality) however whenever I try to test my model using the model.test() function I get the following error.

RuntimeError Traceback (most recent call last)

in 1 # print(model.test(test_data, k=10)) ----> 2 model.test(test_data) //anaconda3/envs/grainger/lib/python3.6/site-packages/extremeText/ExtremeText.py in test(self, path, k) 217 def test(self, path, k=1): 218 """Evaluate supervised model using file given by path""" --> 219 return self.x.test(path, k) 220 221 def quantize( RuntimeError: Encountered NaN.

The training function runs fine and I am also able to use the predict function but am unable to test the accuracy of my model. Following is the training function.

model = train_supervised(input = train_data, epoch = 20, lr = 1, wordNgrams = 5, minn = 1, maxn = 10, loss = 'hs', verbose = 2)

Would you have any idea of what's causing / how to fix this issue so I can benchmark extremetext for our application? This model tests fine in fasttext.

Thanks.

Umar-Ayub commented 5 years ago

A follow up to this. When I try to train a model using the cli I get similar errors where it is unable to open/ close files ./extremetext supervised -input "Dev/data.train" -output model -epoch 100

Reading input file ... libc++abi.dylib: terminating with uncaught exception of type std::invalid_argument: Dev/data.train cannot be opened for training! Abort trap: 6

Umar-Ayub commented 5 years ago

Actually I needed to rollback one version of fasttext and that fixed the issue for me.