TypeError: dispatch_model() got an unexpected keyword argument 'offload_index'

[alisyzhu]

base_model='llama-7b-hf', 2 GPU, but failed.

[LZY-the-boys]

It seems like the errors from internal packages (Maybe Accelerate?) . May you try to update these package to newest first ?

pip install git+https://github.com/huggingface/transformers.git
pip install git+https://github.com/huggingface/peft.git
pip install -U accelerate

Also please attach the full traceback for us to recognize the exact error.

[alisyzhu]

yes，thank you ~