AttributeError: 'Trainer' object has no attribute 'is_world_master'

blagav commented 3 years ago

Environment info

transformers version: 4.0.0
Platform: Linux-4.15.0-96-generic-x86_64-with-debian-buster-sid
Python version: 3.7.8
PyTorch version (GPU?): 1.7.0 (True)
Tensorflow version (GPU?): not installed (NA)
Using GPU in script?: Yes
Using distributed or parallel set-up in script?: No
Ray version: 1.0.1.post1

Who can help

@sgugger — Would you be able to offer any insight?

Information

Model I am using (Bert, XLNet ...): BertForSequenceClassification

The problem arises when using:

[ ] the official example scripts:
[ x] my own modified scripts:

The tasks I am working on is:

[ ] an official GLUE/SQUaD task:
[ x] my own task or dataset:

To reproduce

The error occurs when I try to run a hyperparameter search for my finetuning step using Ray Tune. I am able to successfully finetune the BertForSequenceClassification model normally — the error only arises when running hyperparameter search.

config = BertConfig.from_pretrained(pretrained_model_path, num_labels=num_labels, finetuning_task ='text-classification')

def model_init():
    return BertForSequenceClassification.from_pretrained(pretrained_model_path, config=config)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    model_init=model_init
)

trainer.hyperparameter_search(
    direction="minimize", 
    backend="ray",
    n_trials=20,
    keep_checkpoints_num=1,
    resources_per_trial = {'gpu':1, 'cpu':1}
)

Expected behavior

I am trying to run Ray Tune from Huggingface as per these instructions: https://huggingface.co/blog/ray-tune

If anyone has any insight as to what could be causing this error, it would be greatly appreciated, thank you!

15810856129 commented 3 years ago

you may replace is_world_master by is_world_process_zero.

persas commented 3 years ago

Any solutions to this? I am not able to use ray tune backend in any way

LysandreJik commented 3 years ago

If you get this error still it's probably because you're using mismatched library versions/example versions. Using the latest examples or a more recent version (v4.1.x) should patch this.

If it doesn't, then please open a new issue and fill in the issue template so that we may help you. Thank you.

huggingface / transformers