plasticityai / magnitude

A fast, efficient universal vector embedding utility package.
MIT License
1.63k stars 120 forks source link

Error Loading ELMo #45

Open jsedoc opened 5 years ago

jsedoc commented 5 years ago

I'm just trying to use the ELMo embedding feature and getting the following issue:

using a fresh pip install in a virtualenv (D) jsedoc@****:$ pip install -U pymagnitude Requirement already up-to-date: pymagnitude in /home/jsedoc/venvs/D/lib/python3.6/site-packages Requirement already up-to-date: numpy>=1.14.0 in /home/jsedoc/venvs/D/lib/python3.6/site-packages (from pymagnitude) Requirement already up-to-date: xxhash>=1.0.1 in /usr/local/lib/python3.6/site-packages (from pymagnitude) Requirement already up-to-date: fasteners>=0.14.1 in /usr/local/lib/python3.6/site-packages (from pymagnitude) Requirement already up-to-date: annoy>=1.11.4 in /home/jsedoc/venvs/D/lib/python3.6/site-packages (from pymagnitude) Requirement already up-to-date: lz4>=1.0.0 in /home/jsedoc/venvs/D/lib/python3.6/site-packages (from pymagnitude) Requirement already up-to-date: h5py>=2.8.0 in /home/jsedoc/venvs/D/lib/python3.6/site-packages (from pymagnitude) Requirement already up-to-date: torch in /home/jsedoc/venvs/D/lib/python3.6/site-packages (from pymagnitude) Requirement already up-to-date: monotonic>=0.1 in /usr/local/lib/python3.6/site-packages (from fasteners>=0.14.1->pymagnitude) Requirement already up-to-date: six in /home/jsedoc/venvs/D/lib/python3.6/site-packages (from fasteners>=0.14.1->pymagnitude) You are using pip version 9.0.1, however version 18.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command.

`(D) jsedoc@:$ python Python 3.6.4 (default, Jan 22 2018, 23:35:54) [GCC 4.8.5] on linux Type "help", "copyright", "credits" or "license" for more information.

from pymagnitude import * w2v_vecs = Magnitude('/data1/embeddings/pymagnitude/GoogleNews-vectors-negative300.magnitude') elmo_vecs = Magnitude('/data1/embeddings/pymagnitude/elmo_2x4096_512_2048cnn_2xhighway_5.5B_weights_GoogleNews_vocab.magnitude') Traceback (most recent call last): File "", line 1, in File "/home/jsedoc/venvs/D/lib/python3.6/site-packages/pymagnitude/init.py", line 361, in init .fetchall()[0][0] IndexError: list index out of range`

am05mhz commented 5 years ago

got list index out of range too only by loading the model and try the similarity function

model = Magnitude('model.path.magnitude')
model.similarity('knapsack', 'bag')

btw i'm using the ELMo model data http://magnitude.plasticity.ai/elmo/heavy/elmo_2x4096_512_2048cnn_2xhighway_5.5B_weights_GoogleNews_vocab.magnitude

note: since the models are large files, maybe provide a checksum, so that we can check if our download is corrupt or not

NullP0interEx commented 5 years ago

got list index out of range too only by loading the model and try the similarity function

model = Magnitude('model.path.magnitude')
model.similarity('knapsack', 'bag')

btw i'm using the ELMo model data http://magnitude.plasticity.ai/elmo/heavy/elmo_2x4096_512_2048cnn_2xhighway_5.5B_weights_GoogleNews_vocab.magnitude

note: since the models are large files, maybe provide a checksum, so that we can check if our download is corrupt or not

Same here!

Traceback (most recent call last): File "test.py", line 3, in <module> elmo_vecs = Magnitude('/root/elmo_2x4096_512_2048cnn_2xhighway_5.5B_weights_GoogleNews_vocab.magnitude') File "/usr/local/lib/python3.6/dist-packages/pymagnitude/__init__.py", line 361, in __init__ .fetchall()[0][0] IndexError: list index out of range

khadkechetan commented 4 years ago

Same here. Got the same error.