Add fsdp+qlora support - Githubissues

huggingface / alignment-handbook

Robust recipes to align language models with human and AI preferences

https://huggingface.co/HuggingFaceH4

Apache License 2.0

4.53k stars 393 forks source link

Add fsdp+qlora support #160

Closed deep-diver closed 4 months ago

deep-diver commented 5 months ago

This PR is to add FSDP+QLoRA support with the following changes:

add recipes/accelerate_configs/fsdp+qlora.yaml
update versions of peft>=0.9.0 and bitsandbytes>=0.43.0 dependencies
bnb_4bit_quant_storage field in ModelArguments
set bnb_4bit_quant_storage of BitsAndBytesConfig

With these changes, I have confirmed FSDP+QLoRA works within my local setup (2 x A6000).

deep-diver commented 5 months ago

@lewtun

Besides keeping or removing fsdp+qlora.yaml discussion, I made additional commit for adding an example on https://github.com/huggingface/alignment-handbook/tree/main/scripts#fine-tuning. Please take a look!

HuggingFaceDocBuilderDev commented 5 months ago

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

deep-diver commented 5 months ago

@lewtun

reminder

deep-diver commented 4 months ago

@lewtun

reminder. I addressed your comments :)