dep: cap transformers version due to FSDP bug

foundation-model-stack / fms-hf-tuning

🚀 Collection of tuning recipes with HuggingFace SFTTrainer and PyTorch FSDP.

Apache License 2.0

28 stars 48 forks source link

dep: cap transformers version due to FSDP bug #335

Closed anhuong closed 2 months ago

anhuong commented 2 months ago

Description of the change

v0.34.0 and v0.34.1 of accelerate has a bug where when training with FSDP, the weight is being flattened and is unable to be loaded. Capping at v0.33 before the change went in and at a verified point.

Related issue number

How to verify the PR

Was the PR tested

[ ] I have added >=1 unit test(s) for every new method I have added.
[ ] I have ensured all unit tests pass