search

google-research / xtreme

XTREME is a benchmark for the evaluation of the cross-lingual generalization ability of pre-trained multilingual models that covers 40 typologically diverse languages and includes nine tasks.
https://sites.research.google/xtreme
Apache License 2.0
631 stars 110 forks source link

2513 Segmentation fault: 11 when running Tatoeba #86

Open vrmer opened 2 years ago

vrmer commented 2 years ago

When I attempt to run bash scripts/train.sh "bert-base-multilingual-cased" tatoeba, I receive the following segmentation fault after loading the model config:

/Users/marcellfekete/PycharmProjects/xtreme/scripts/run_tatoeba.sh: line 50:  2513 Segmentation fault: 11  python $REPO/third_party/evaluate_retrieval.py --model_type $MODEL_TYPE --model_name_or_path $MODEL --embed_size $DIM --batch_size 100 --task_name $TASK --src_language $SL --tgt_language en --data_dir $DATA_DIR/$TASK/ --max_seq_length $MAXL --output_dir $OUT --log_file embed-cosine --num_layers $NLAYER --dist cosine $LC --specific_layer $LAYER
/Users/marcellfekete/miniforge3/envs/rosetta/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

Do you have any idea what could cause this and what can be done to fix it? Thank you!

sebastianruder commented 2 years ago

Hi Marcell, I haven't seen this before I'm afraid. I'll let you know once I've had time to reproduce the issue.