run examples/llama-2/qlora-fsdp.yml failed

[XiepengLi]

Please check that this issue hasn't been reported before.

• [X] I searched previous Bug Reports didn't find any similar reports.

Expected Behavior

should start training

Current behaviour

site-packages/torch/nn/modules/module.py", line 1688, in __getattr__
    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'LlamaForCausalLM' object has no attribute 'hf_quantizer'

Steps to reproduce

accelerate launch -m axolotl.cli.train examples/llama-2/qlora-fsdp.yml

Config yaml

No response

Possible solution

No response

Which Operating Systems are you using?

• [X] Linux
• [ ] macOS
• [ ] Windows

Python Version

3.12

axolotl branch-commit

main

Acknowledgements

• [X] My issue title is concise, descriptive, and in title casing.
• [X] I have searched the existing issues to make sure this bug has not been reported yet.
• [X] I am using the latest version of axolotl.
• [X] I have provided enough information for the maintainers to reproduce and diagnose the issue.

[NanoCode012]

Hey, does this issue still occur on latest main with upto date dependency?

[XiepengLi]

ValueError: Must flatten tensors with uniform dtype but got torch.float32 and torch.float16

[accupham]

ValueError: Must flatten tensors with uniform dtype but got torch.float32 and torch.float16

Have same issue too.