TRL/Alignment-Handbook torch.dtype Issues

huggingface / alignment-handbook

Robust recipes to align language models with human and AI preferences

https://huggingface.co/HuggingFaceH4

Apache License 2.0

4.18k stars 354 forks source link

TRL/Alignment-Handbook torch.dtype Issues #175

Open neelsjain opened 1 week ago

neelsjain commented 1 week ago

I believe that the torch.dtype handling is currently broken in TRL, which is causing issues with alignment handbook. I was able to get around this by downgrading to the TRL version (0.8.2) when this repo was created.