search

twadada / multilingual-nlm

Code for "Unsupervised Multilingual Word Embedding with Limited Resources using Neural Language Models" and "Learning Contextualised Cross-lingual Word Embeddings and Alignments for Extremely Low-Resource Languages Using Parallel Corpora"
30 stars 5 forks source link

NoneType error when trying to train DE-EN model #3

Open dsli208 opened 2 years ago

dsli208 commented 2 years ago

I'm trying to follow the instructions for reproducing the DE-EN word alignment experiment, but I am hitting a snag when trying to run train.py:

Traceback (most recent call last):
  File "train.py", line 22, in <module>
    opt.save_dir = opt.save_dir +'-'.join([str(x) for x in opt.lang_class])
TypeError: 'NoneType' object is not iterable

Is there an issue with the file that I should be aware of, or is there a quick fix for such an issue? I ran the --help flag and I'm apparently missing a LANG_CLASS, is that provided as part of reproducing said experiment?

twadada commented 2 years ago

Adding the option "-lang_class 0 1" should solve the issue. Let me know if you've got any further questions.

dsli208 commented 2 years ago

When I try this, a new issue will arise when I try and run train.py:

RuntimeError: CUDA out of memory. Tried to allocate 7.06 GiB (GPU 0; 11.91 GiB total capacity; 3.55 GiB already allocated; 6.99 GiB free; 3.99 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
twadada commented 2 years ago

That's an OOM error. You may either use a GPU with larger memories or use multiple GPUs (e.g. by setting CUDA_VISIBLE_DEVICES=0,1)

dsli208 commented 2 years ago

issue still seems to persist after i try this

twadada commented 2 years ago

In that case, you may use more GPUs (CUDA_VISIBLE_DEVICES=0,1,2). Alternatively, before you apply the following preprocess:

python filter_long_sent.py -src ${de} -tgt ${en}

you may reduce the cut-off threshold in "filter_long_sent.py"

if len(src[i].split())<=80 and len(tgt[i].split())<=80:

from 80 to 70/60 to remove long sentences, and run python preprocess.py again. Let me know if it doesn't still go well. I appreciate your patience.