Open deep-diver opened 2 months ago
For QLoRA+FSDP support, the dependencies should be updated:
bitsandbytes>=0.43.0
accelerate>=0.28.0
transformers>4.38.2
trl>0.7.11
peft>0.9.0
Also, it would be wonderful to have accelerate recipe for this too.
For QLoRA+FSDP support, the dependencies should be updated:
bitsandbytes>=0.43.0accelerate>=0.28.0transformers>4.38.2trl>0.7.11peft>0.9.0Also, it would be wonderful to have accelerate recipe for this too.