TypeError: __init__() got an unexpected keyword argument 'report_to'

TatProg commented 3 years ago

Environment info

transformers version: 4.6.1
Platform: Linux-4.15.0-135-generic-x86_64-with-Ubuntu-18.04-bionic
Python version: 3.6.9
PyTorch version (GPU?): 1.9.0+cu102 (True)
Tensorflow version (GPU?): not installed (NA)
Using GPU in script?: Yes, 8 Tesla V100
Using distributed or parallel set-up in script?: training_args.parallel_mode = ParallelMode.NOT_DISTRIBUTED

Who can help

@sgugger @LysandreJik

Information

Model I am using BERT - 'bert-base-multilingual-cased'

The problem arises when using:

[x] the official example scripts.

The tasks I am working on is:

[x] an official GLUE/SQUaD task: Masked Language Modelling

To reproduce

Steps to reproduce the behavior:

Install wandb and tensorboard via Command Line


pip install wandb
wandb login
WANDB_PROJECT=mlm_project

pip install tf-nightly pip install tensorboard

1. Initialize training_args
```python
training_args = TrainingArguments(
    output_dir='My_lovely_mlm_model',
    overwrite_output_dir=True,
    do_train=True,
    do_eval=True,
    per_device_train_batch_size=100,
    per_device_eval_batch_size=50,
    evaluation_strategy='steps',
    logging_steps=10_000,
    eval_steps=None,
    prediction_loss_only=True,
    num_train_epochs=50,
    save_steps=10_000,
    save_total_limit=10,

    report_to=['wandb', 'tensorboard']

)

Run script with button in PyCharm or in console with python run_mlm.py --report_to wandb --run_name new_run_name

Enjoy error message

Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/Users/LysandreJik_is_the_best/Documents/PyCharmProjects/in_sgugger_we_trust/mlm/run_mlm.py", line 21, in <module>
from arguments_for_mlm import model_data_args
File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
File "/Users/LysandreJik_is_the_best/Documents/PyCharmProjects/in_sgugger_we_trust/mlm/arguments_for_mlm.py", line 299, in <module>
report_to=['wandb', 'tensorboard']
TypeError: __init__() got an unexpected keyword argument 'report_to'

Also, i tried to remove 'tensorboard' or 'wandb', but caught same error again and again

Expected behavior

Script must run without this error. Wandb folder must be created.

LysandreJik commented 3 years ago

Hello! Do you mind sharing a reproducible code example in colab? I ran the following on the bare run_mlm.py script:

python run_mlm.py --output_dir=here --report_to=tensorboard --dataset_name=wikitext --dataset_config_name=wikitext-2-raw-v1 --model_name_or_path=bert-base-cased --do_train

which works without issue. Maybe @sgugger has more insights.

Nice username and Pycharm project :)

TatProg commented 3 years ago

@LysandreJik

I found problem: I used report_to= in TrainingArguments without run_name=

Even if I add --run_name= in command python3 run_mlm.py --report_to wandb --run_name new_run_name it didn't work. I think this might be fixed quite easy: Throw exception, if report_to= is 'initialized' in TrainingArguments without run_name=

My solution. Also tested not only with 'wandb', but with 'tensorboard' and both of them works very well. (All imports are above)

training_args = TrainingArguments(
    output_dir='My_lovely_mlm_model',
    overwrite_output_dir=True,
    do_train=True,
    do_eval=True,
    per_device_train_batch_size=100,
    per_device_eval_batch_size=50,
    evaluation_strategy='steps',
    logging_steps=10_000,
    eval_steps=None,
    prediction_loss_only=True,
    num_train_epochs=50,
    save_steps=10_000,
    save_total_limit=10,

    report_to='wandb'
    run_name="new_run" #ADDED THIS.

)

P.S. If I want to continue training my model from checkpoint, i should change model = AutoModelForMaskedLM.from_pretrained('bert-base-multilingual-cased') to model = AutoModelForMaskedLM.from_pretrained('project/pretrained_model_name/checkpoint-190000')right?

sgugger commented 3 years ago

If you use the Trainer, just use resume_from_checkpoint=path_to_checkpoint when calling trainer.train.

huggingface / transformers