search

pytorch / torchtitan

A native PyTorch Library for large model training
BSD 3-Clause "New" or "Revised" License
1.28k stars 115 forks source link

Expose mixed_precision dtype arguments #348

Closed wconstab closed 1 month ago

wconstab commented 1 month ago

Stack from ghstack (oldest at bottom):

add training.mixed_precision_param and .mixed_precision_reduce options

refactor a util to map strings to torch dtypes