search

lanwuwei / SPM_toolkit

Neural network toolkit for sentence pair modeling.
303 stars 70 forks source link

load_word_vectors problems #32

Open funnytestingcase opened 3 years ago

funnytestingcase commented 3 years ago

I have torchtext 0.1.1 , python 2.7 but function "load_word_vectors" would not download the correct zip file (the 2.18GB one) when it starts downloading "glove.840B.300d: 8.19kB [00:00, 9.51kB/s]" and it always ends up with a bad zip file

was I missing anything?

lanwuwei commented 3 years ago

torchtext 0.1.1 is correct, how about your network speed?

funnytestingcase commented 3 years ago

I solved this error by modifying vocab.py file from torchtext, basically I bypassed the downloading the unzip step.

and, What's your memory size when implementing those four frameworks? My 4GB was quickly run out.

lanwuwei commented 3 years ago

4GB is too small, I ran this code in a server with tens of GB of memory. ESIM is memory efficient, as it uses the file iterator without loading all the data into memory. You can try ESIM first with your 4GB memory.

funnytestingcase commented 3 years ago

thx for the quick reply !