TFknight
commented
6 years ago try to train with small batch_size, small persentage of dev_samples. I hope this help you~
Open swathimithran opened 7 years ago
TFknight
commented
6 years ago try to train with small batch_size, small persentage of dev_samples. I hope this help you~
Error while running glove_model.py file
I am supplying a corpus to train, I have Copied corpus in Glove-1.2 directory and ran the code from embeddings folder and gave the name of corpus as param But I get the following error,
Stacktrace : sudo python glove_model.py --corpus /home/swathimithran/workplace/NER-ISTM/ner-lstm-master/thirdparty/GloVe-1.2/english.txt --dimension 111 --glove_path /home/swathimithran/workplace/NER-ISTM/ner-lstm-master/thirdparty/GloVe-1.2/
mkdir -p build gcc src/glove.c -o build/glove -lm -pthread -Ofast -march=native -funroll-loops -Wno-unused-result gcc src/shuffle.c -o build/shuffle -lm -pthread -Ofast -march=native -funroll-loops -Wno-unused-result gcc src/cooccur.c -o build/cooccur -lm -pthread -Ofast -march=native -funroll-loops -Wno-unused-result gcc src/vocab_count.c -o build/vocab_count -lm -pthread -Ofast -march=native -funroll-loops -Wno-unused-result BUILDING VOCABULARY Processed 141 tokens. Counted 103 unique words. Truncating vocabulary at min count 5. Using vocabulary of size 4.
COUNTING COOCCURRENCES window size: 15 context: symmetric max product: 13752509 overflow length: 38028356 Reading vocab from file "vocab.txt"...loaded 4 words. Building lookup table...table contains 17 elements. Processed 141 tokens. Writing cooccurrences to disk..2 files in total. Merging cooccurrence files: processed 17 lines.
SHUFFLING COOCCURRENCES array size: 255013683 Shuffling by chunks: processed 17 lines. Wrote 1 temporary file(s). Merging temp files: processed 17 lines.
TRAINING MODEL Read 17 lines. Initializing parameters...done. vector size: 111 vocab size: 4 x_max: 10.000000 alpha: 0.750000 demo.sh: line 35: 11029 Segmentation fault (core dumped) $BUILDDIR/glove -save-file $SAVE_FILE -threads $NUM_THREADS -input-file $COOCCURRENCE_SHUF_FILE -x-max $X_MAX -iter $MAX_ITER -vector-size $VECTOR_SIZE -binary $BINARY -vocab-file $VOCAB_FILE -verbose $VERBOSE Traceback (most recent call last): File "glove_model.py", line 48, in
model = GloveVec(args)
File "glove_model.py", line 21, in init
for line in open(path):
IOError: [Errno 2] No such file or directory: 'vectors.txt'