taishi-i / nagisa

A Japanese tokenizer based on recurrent neural networks
https://huggingface.co/spaces/taishi-i/nagisa-demo
MIT License
379 stars 22 forks source link

Illegal instruction (core dumped) #26

Closed paulm17 closed 3 years ago

paulm17 commented 3 years ago

Thanks for building this. I've been trying mecab and not been getting the exact results that I need and thought I'd give this a try.

For now, I have this working on a centos box, but I'm wanting to get this working on ubuntu as it's my main dev machine.

I keep getting:

[dynet] random seed: 1234
[dynet] allocating memory: 32MB
Illegal instruction (core dumped)

Distributor ID: Ubuntu Description: Ubuntu 20.04 LTS Release: 20.04 Codename: focal

Is there any more information you need? Thanks

taishi-i commented 3 years ago

Hi @paulm17. Thank you for using nagisa! I have tested it on my Ubuntu environment (Ubuntu 20.04 LTS). I used pyenv with Python 3.8.5, as shown below. It seems to work fine.

$ pyenv install 3.8.5
$ mkdir work
$ cd work
$ pyenv local 3.8.5 
$ pip install nagisa
$ python

This is the execution result on my Ubuntu environment.

Python 3.8.5 (default, Nov 19 2020, 09:33:42) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import nagisa
[dynet] random seed: 1234
[dynet] allocating memory: 32MB
[dynet] memory allocation done.
>>> tokens = nagisa.tagging("これはテストです")
>>> print(tokens)
これ/代名詞 は/助詞 テスト/名詞 です/助動詞

As a solution to the current situation, could you install directly from the source? If you still get an error, could you give me some information on the Python interpreter?

pip uninstall nagisa
pip install git+https://github.com/taishi-i/nagisa

Thank you.

paulm17 commented 3 years ago

Thanks for replying, appreciate it.

I'm still getting core dumped, is there a minimum memory requirement for nagisa?

Python 3.8.5 (default, Jul 28 2020, 12:59:40)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import nagisa
[dynet] random seed: 1234
[dynet] allocating memory: 32MB
Illegal instruction (core dumped)

Thanks