search

pytorch / torchtitan

A native PyTorch Library for large model training
BSD 3-Clause "New" or "Revised" License
2.25k stars 165 forks source link

Set `record_shapes=True` for profiler #419

Closed awgu closed 3 months ago

awgu commented 3 months ago

Stack from ghstack (oldest at bottom):

record_shapes=True should have negligible extra runtime overhead and does not bloat the trace file. We should set it since it helps understanding traces.