pytorch / torchdynamo

A Python-level JIT compiler designed to make unmodified PyTorch programs faster.
BSD 3-Clause "New" or "Revised" License
1k stars 123 forks source link

[inductor] TIMM training failures tracker #780

Closed desertfire closed 2 years ago

desertfire commented 2 years ago

[Editied]

benchmarks/timm_models.py -d cuda --inductor --training --float32 --use-eval-mode

Snapshot of Aug 22:

The following ones also fail with aot_nvfuser:

eellison commented 2 years ago

I got errors for both convnext_base and xcit_large_24_p8_224 when running with --accuracy-aot-nop.

legacy_senet154 also fails in eager - Variation in Eager runs itself

desertfire commented 2 years ago

I got errors for both convnext_base and xcit_large_24_p8_224 when running with --accuracy-aot-nop.

legacy_senet154 also fails in eager - Variation in Eager runs itself

Thanks for checking. Do you mind to file separate issues to track those for closer attention to folks working on AOTAutograd?

desertfire commented 2 years ago

Combine with https://github.com/pytorch/pytorch/issues/93777