pytorch / torchtitan

A native PyTorch Library for large model training
BSD 3-Clause "New" or "Revised" License
2.25k stars 165 forks source link

whole_model for fp8 #414

Closed weifengpy closed 3 months ago

weifengpy commented 3 months ago

train.py renamed model to whole_model https://github.com/pytorch/torchtitan/pull/406

fp8 still use model thus report error on model not defined. this PR fixed it

build_fp8_linear(whole_model, job_config)