search

pytorch / torchtitan

A native PyTorch Library for large model training
BSD 3-Clause "New" or "Revised" License
2.68k stars 212 forks source link

Control 'deterministic' mode separately from rng seed #690

Closed wconstab closed 6 days ago

wconstab commented 6 days ago

Stack from ghstack (oldest at bottom):

The use case for 'deterministic' mode may be more for debugging, while users may want to control RNG seeds used for real runs.

wconstab commented 6 days ago

squashed into previous PR