Open XiepengLi opened 6 months ago
Hey, does this issue still occur on latest main with upto date dependency?
ValueError: Must flatten tensors with uniform dtype but got torch.float32 and torch.float16
ValueError: Must flatten tensors with uniform dtype but got torch.float32 and torch.float16
Have same issue too.
Please check that this issue hasn't been reported before.
Expected Behavior
should start training
Current behaviour
Steps to reproduce
accelerate launch -m axolotl.cli.train examples/llama-2/qlora-fsdp.yml
Config yaml
No response
Possible solution
No response
Which Operating Systems are you using?
Python Version
3.12
axolotl branch-commit
main
Acknowledgements