segmentation fault when predicting

[rola93]

I have implemented a python class supporting sckit learn's interfaces to train my predictor using sckit learn's grid search.

However, after running some time it throws this:

File "/home/rodrigo/FastTextClassifier.py", line 114, in predict classified = self.model.predict(X) File "src/pyfasttext.pyx", line 782, in pyfasttext.FastText.predict File "src/pyfasttext.pyx", line 767, in pyfasttext.FastText.predict_aux cysignals.signals.SignalError: Segmentation fault

Any idea of what is happening? I'm trying to save the X which breaks it all to use from command line, but its not easy since sckit learn's grid search invokes it multiple times with different parameters, and this error is not throwing any exception, just finishes the process

[vrasneur]

Hum... Can you generate a coredump?

[rola93]

Mmmm I'm not sure.... What is exactly a coredump? how do I do it?

[vrasneur]

It is a memory image of the program, that is generated when the crash happened.

It is useful when you debug the crash: you can see what the memory looked like at that time.

Here is a tutorial on how to generate it (it's easy!): http://bencane.com/2011/09/22/kill-creating-a-core-dump/

[bung87]

same issues here,I use fastText and pyfasttext do same processing. the model has 3.0G bandwidth. I'm working on 2.7 GHz Intel Core i5 and 8 GB 1867 MHz DDR3 mac book. fastText Traceback: Exception: fastText: Cannot load sketch/fasttext/../../data/wiki.zh.bin due to C++ extension failed to allocate the memory

pyfasttext Traceback:

Traceback (most recent call last): File "sketch/fasttext/main.py", line 17, in r = model.predict_proba_single('{}\n'.format(sys.argv[1]), k=2) File "src/pyfasttext.pyx", line 785, in pyfasttext.FastText.predict_proba_single File "src/pyfasttext.pyx", line 779, in pyfasttext.FastText.predict_proba File "src/pyfasttext.pyx", line 767, in pyfasttext.FastText.predict_aux cysignals.signals.SignalError: Segmentation fault

0 signals.cpython-35m-darwin.so 0x0000000108a8ab98 sigdie + 120 1 signals.cpython-35m-darwin.so 0x0000000108a8aacf cysigs_signal_handler + 351 2 libsystem_platform.dylib 0x00007fff8615352a _sigtramp + 26 3 ??? 0xfffffffffff0bdc0 0x0 + 18446744073708551616 4 pyfasttext.cpython-35m-darwin.so 0x0000000107b0bb22 _ZN8fasttext10DictionaryD2Ev + 194 5 libc++.1.dylib 0x00007fff82784cb8 _ZNSt3119shared_weak_count16__release_sharedEv + 44 6 pyfasttext.cpython-35m-darwin.so 0x0000000107af180d _ZN8fasttext8FastTextD2Ev + 93 7 pyfasttext.cpython-35m-darwin.so 0x0000000107af0ee0 _ZL38__pyx_tp_dealloc_10pyfasttext_FastTextP7_object + 48 8 Python 0x000000010749d16e free_keys_object + 81 9 Python 0x000000010749e5e5 dict_dealloc + 199 10 Python 0x00000001074a8052 module_dealloc + 115 11 Python 0x000000010749c5bc insertdict + 390 12 Python 0x0000000107521630 PyImport_Cleanup + 591 13 Python 0x000000010752b9e3 Py_Finalize + 98 14 Python 0x0000000107541f72 Py_Main + 3566 15 python 0x000000010745ee3d python + 7741 16 libdyld.dylib 0x00007fff97a555ad start + 1

Unhandled SIGSEGV: A segmentation fault occurred. This probably occurred because a compiled module has a bug in it and is not properly wrapped with sig_on(), sig_off(). Python will now terminate.

Segmentation fault: 11

[rola93]

I could not solve that problem. Finally I end up using a python interface provided by fasttext (I don't know how did'nt I see it at first time). It is in the fasttext project, on python folder.

I didn't found lots of documentation but after playing a bit with ipython and with the knolodge I get from fast text command line tool I could use it.

According to their doc, the easiest way to install fastText is to use pip.

$ git clone https://github.com/facebookresearch/fastText.git
$ cd fastText
$ pip install . 

(don't forget the dot after pip install)

You can also pip install it directly without cloning the repo, which is really faster, with the following command:

pip install git+https://github.com/facebookresearch/fastText

Hope this helps!

[bung87]

related issue from fastText.py which the lib named fasttext through pip
https://github.com/salestock/fastText.py/issues/115

This error is mainly because the wiki.en.bin file is trained with newer version of fastText, and as of now you can't use current fastText.py python library to load/use newer models.

use the facebook official version is the right choise!