search

ccsasuke / umwe

Unsupervised Multilingual Word Embeddings (EMNLP 2018)
https://arxiv.org/abs/1808.08933
Other
81 stars 17 forks source link

Seg fault in eval with faiss-gpu #2

Closed mjmartindale closed 6 years ago

mjmartindale commented 6 years ago

I'm trying to run unsupervised.py, and I consistently get a seg fault in the word translation evaluation unless I drop faiss. The version of faiss I'm using is:

Name Version Build Channel faiss-gpu 1.4.0 py36_cuda8.0.61_1 pytorch

The command I'm running (with few languages and small epoch size in the hope that it might run then) is:

python unsupervised.py --src_langs es fr --tgt_lang en --epoch_size 10000 --exp_path /tmp/umwe_190103/ --emb_dir /tmp/umwe_190103/umwe-data/fasttext_vectors --dico_eval /tmp/umwe_190103/umwe-data/crosslingual/dictionaries

And the error looks like:

INFO - 10/16/18 20:01:17 - 0:01:06 - Found 2416 pairs of words in /tmp/umwe_190103/umwe-data/crosslingual/dictionaries/es-en.5000-6500.txt (1500 unique). 0 other pairs contained at least one unknown word (0 in lang1, 0 in lang2) unsupervised2.sh: line 36: 190190 Segmentation fault (core dumped) python unsupervised.py --src_langs es fr --tgt_lang en --epoch_size 10000 --exp_path /tmp/umwe_190103/ --emb_dir /tmp/umwe_190103/umwe-data/fasttext_vectors --dico_eval /tmp/umwe_190103/umwe-data/crosslingual/dictionaries

Any suggestions welcome!

ccsasuke commented 6 years ago

Hi,

I have not seen this issue before.

The faiss version I was using was a slightly older one but with a newer CUDA library: version 1.3.0, py36_cuda9.0.176_1 You can try that to see if it fixes your problem. If this does not fix the error and it works fine without faiss, maybe try to contact the faiss support?

mjmartindale commented 6 years ago

Downgrading faiss and upgrading cuda seems to have worked, thanks!