search

monikkinom / ner-lstm

Named Entity Recognition using multilayered bidirectional LSTM
539 stars 183 forks source link

Glove error vector.txt not found #24

Open swathimithran opened 7 years ago

swathimithran commented 7 years ago

Error while running glove_model.py file

I am supplying a corpus to train, I have Copied corpus in Glove-1.2 directory and ran the code from embeddings folder and gave the name of corpus as param But I get the following error,

Stacktrace : sudo python glove_model.py --corpus /home/swathimithran/workplace/NER-ISTM/ner-lstm-master/thirdparty/GloVe-1.2/english.txt --dimension 111 --glove_path /home/swathimithran/workplace/NER-ISTM/ner-lstm-master/thirdparty/GloVe-1.2/

mkdir -p build gcc src/glove.c -o build/glove -lm -pthread -Ofast -march=native -funroll-loops -Wno-unused-result gcc src/shuffle.c -o build/shuffle -lm -pthread -Ofast -march=native -funroll-loops -Wno-unused-result gcc src/cooccur.c -o build/cooccur -lm -pthread -Ofast -march=native -funroll-loops -Wno-unused-result gcc src/vocab_count.c -o build/vocab_count -lm -pthread -Ofast -march=native -funroll-loops -Wno-unused-result BUILDING VOCABULARY Processed 141 tokens. Counted 103 unique words. Truncating vocabulary at min count 5. Using vocabulary of size 4.

COUNTING COOCCURRENCES window size: 15 context: symmetric max product: 13752509 overflow length: 38028356 Reading vocab from file "vocab.txt"...loaded 4 words. Building lookup table...table contains 17 elements. Processed 141 tokens. Writing cooccurrences to disk..2 files in total. Merging cooccurrence files: processed 17 lines.

SHUFFLING COOCCURRENCES array size: 255013683 Shuffling by chunks: processed 17 lines. Wrote 1 temporary file(s). Merging temp files: processed 17 lines.

TRAINING MODEL Read 17 lines. Initializing parameters...done. vector size: 111 vocab size: 4 x_max: 10.000000 alpha: 0.750000 demo.sh: line 35: 11029 Segmentation fault (core dumped) $BUILDDIR/glove -save-file $SAVE_FILE -threads $NUM_THREADS -input-file $COOCCURRENCE_SHUF_FILE -x-max $X_MAX -iter $MAX_ITER -vector-size $VECTOR_SIZE -binary $BINARY -vocab-file $VOCAB_FILE -verbose $VERBOSE Traceback (most recent call last): File "glove_model.py", line 48, in model = GloveVec(args) File "glove_model.py", line 21, in init for line in open(path): IOError: [Errno 2] No such file or directory: 'vectors.txt'

TFknight commented 6 years ago

try to train with small batch_size, small persentage of dev_samples. I hope this help you~