run_squad.py not working on 3.1.0 version

[deepanshu650]

Environment info

• transformers version: 3.0.2
• Platform: Linux-4.15.0-91-generic-x86_64-with-debian-buster-sid
• Python version: 3.7.6
• PyTorch version (GPU?): 1.5.0 (False)
• Tensorflow version (GPU?): 2.1.0 (False)
• Using GPU in script?: True
• Using distributed or parallel set-up in script?: True

Who can help

@LysandreJik @sshleifer ## Information Model I am using (Bert, XLNet ...): Bert The problem arises when using: * [x] the official example scripts: (give details below) run_squad.py is working with 2.9.1 but when I update it to 3.1.0 it gives error * [ ] my own modified scripts: (give details below) The tasks I am working on is: * [x] an official GLUE/SQUaD task: SQUaD * [ ] my own task or dataset: (give details below) ## To reproduce Steps to reproduce the behavior: 1. !pip install transformers==3.1.0 It works when using 2.9.1 but not with this 2. !mkdir dataset \ && cd dataset \ && wget https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v2.0.json \ && wget https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json 3. !export SQUAD_DIR=/content/dataset \ && python transformers/examples/run_squad.py \ --model_type bert \ --model_name_or_path bert-base-uncased \ --do_train \ --do_eval \ --do_lower_case \ --train_file $SQUAD_DIR/train-v2.0.json \ --predict_file $SQUAD_DIR/dev-v2.0.json \ --per_gpu_train_batch_size 12 \ --learning_rate 3e-5 \ --num_train_epochs 1.0 \ --max_seq_length 384 \ --doc_stride 128 \ --output_dir /content/model_output \ --save_steps 1000 \ --threads 4 \ --version_2_with_negative Error message is: 2020-09-07 19:33:26.850641: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1 Traceback (most recent call last): File "transformers/examples/run_squad.py", line 74, in (), File "transformers/examples/run_squad.py", line 73, in (tuple(conf.pretrained_config_archive_map.keys()) for conf in (BertConfig, RobertaConfig, XLNetConfig, XLMConfig)), AttributeError: type object 'BertConfig' has no attribute 'pretrained_config_archive_map'

Expected behavior

It should start training without error just like older versions

[LysandreJik]

Indeed, it seems it hasn't been up to date. Did you try running run_squad_trainer.py? It should be more up to date.

@sgugger we should probably deprecate run_squad.py now that we have a Trainer-based SQuAD script.

[julien-c]

The missing part was the eval which shouldn't be too hard to add (https://github.com/huggingface/transformers/pull/4829#issuecomment-645994130)

And then we can rename files as in https://github.com/huggingface/transformers/pull/5582

[deepanshu650]

Indeed, it seems it hasn't been up to date. Did you try running run_squad_trainer.py? It should be more up to date.

@sgugger we should probably deprecate run_squad.py now that we have a Trainer-based SQuAD script.

No run_squad_trainer.py also gives error in 3.1.0 !python transformers/examples/question-answering/run_squad_trainer.py --help Error is python3: can't open file 'transformers/examples/question-answering/run_squad_trainer.py': [Errno 2] No such file or directory

[LysandreJik]

@deepanshu650 the file exists, it's here. Are you sure you cloned the v3.1.0 repo?

[deepanshu650]

Yes it's working , initially I was working on copy of someone's notebook which was installing v2.3.0 and I had to pip install v3.1.0, so after changing to new notebook it starts running. But it doesn't give evaluation results(which run_squad.py does) as run_squad_trainer.py not calling trainer.evaluate() though it makes eval_dataset . So how do I evaluate . Thanks for replying.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.