Investigate numerical error of fp32 resnet18 of `train=True`

Lightning-AI / lightning-thunder

Make PyTorch models up to 40% faster! Thunder is a source to source compiler for PyTorch. It enables using different hardware executors at once; across one or thousands of GPUs.

Apache License 2.0

1.07k stars 60 forks source link

Investigate numerical error of fp32 resnet18 of `train=True` #634

Closed crcrpar closed 4 days ago

crcrpar commented 1 week ago

https://github.com/Lightning-AI/lightning-thunder/blob/9f9dcafc9ba5b07652bbab91a602aec3c628c8d1/thunder/tests/test_inplace_functionalization.py#L152-L158