TypeError: Accelerator.__init__() got an unexpected keyword argument 'dispatch_batches'

bardenthenry commented 6 days ago

System Info

transformers: 4.39.3 python: 3.10.12

Who can help?

No response

Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction

An error may occur at line 580 of run_ner.py

trainer = Trainer(
      model=model,
      args=training_args,
      train_dataset=train_dataset if training_args.do_train else None,
      eval_dataset=eval_dataset if training_args.do_eval else None,
      tokenizer=tokenizer,
      data_collator=data_collator,
      compute_metrics=compute_metrics,
  )

Expected behavior

Traceback (most recent call last):
  File "/usr/src/app/llm_model_test/ner_train/run_ner.py", line 666, in <module>
    main()
  File "/usr/src/app/llm_model_test/ner_train/run_ner.py", line 580, in main
    trainer = Trainer(
  File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 373, in __init__
    self.create_accelerator_and_postprocess()
  File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 4252, in create_accelerator_and_postprocess
    self.accelerator = Accelerator(
TypeError: Accelerator.__init__() got an unexpected keyword argument 'dispatch_batches'

Please Help Me...

SiyuWu528 commented 5 days ago

I meet the same error around the same time. Please Help Us

SiyuWu528 commented 5 days ago

Hi friend, I think I find out the issue. Likely because version mismatch between transformer and accelerate, with your transformer version, try accelerate version 0.28.0

abdelkareemkobo commented 3 days ago

@SiyuWu528 Thanks i faced this problem today and your comment helped me alot

2625009538 commented 3 days ago

@SiyuWu528 Thanks i faced this problem today and your comment helped me alot非常感谢，我今天遇到了这个问题，你的评论帮了我大忙。

What is the version number of your transformers? accelerate Did you set 0.28.0?

mm86443 commented 3 days ago

@SiyuWu528 Thanks i faced this problem today and your comment helped me alot非常感谢，我今天遇到了这个问题，你的评论帮了我大忙。

What is the version number of your transformers? accelerate Did you set 0.28.0?

it's works (base) PS E:> pip list | findstr transform transformers 4.37.2 transformers-stream-generator 0.0.4 (base) PS E:> pip list | findstr accelerate accelerate 0.28.0

abdelkareemkobo commented 2 days ago

transformers==4.37.2 and accelerate==0.28.0 worked on my ubuntu labtop but failed on windows server

huggingface / transformers