search

pytorch / torchdynamo

A Python-level JIT compiler designed to make unmodified PyTorch programs faster.
BSD 3-Clause "New" or "Revised" License
1.01k stars 123 forks source link

[Inductor] [CPU] Accuracy failure in torchbench model Super_SloMo on WW03.2 #2035

Closed ESI-SYD closed 1 year ago

ESI-SYD commented 1 year ago

🐛 Describe the bug

Compare with the ww02.4, there is a accuracy failure on Torchbench model Super_SloMo on TorchInductor CPU Performance Dashboard on ww03.2:

WW03.2 SW info:

SW Nightly commit Master/Main commit
Pytorch 868797a d41b5d7
Torchbench / bbdc777
torchaudio ecc2781 4a037b0
torchtext 112d757 c7cc5fc
torchvision ac06efe 35f68a0
torchdata 049fb62 c0934b9
dynamo_benchmarks fac4361 /

WW02.4 SW info:

SW Nightly commit Master/Main commit
Pytorch 466be3f 364f526
Torchbench / bbdc777
torchaudio ecc2781 4a037b0
torchtext 112d757 c7cc5fc
torchvision ac06efe 35f68a0
torchdata 049fb62 c0934b9
dynamo_benchmarks fac4361 /

Error logs

cpu  eval  Super_SloMo                        [2023-01-16 06:56:05,488] torch._dynamo.utils: [ERROR] RMSE (res-fp64): 0.01816, (ref-fp64): 0.00000 and shape=torch.Size([])
FAIL

Minified repro

python -m torch.backends.xeon.run_cpu --core_list 0 --ncores_per_instance 1 benchmarks/dynamo/torchbench.py --accuracy --float32 -dcpu -n50 --inductor  --no-skip --dashboard --only Super_SloMo  --cold_start_latency --batch_size 1 --threads 1
chunyuan-w commented 1 year ago

The issue has been fixed according to WW03.4 dashboard: https://github.com/pytorch/pytorch/issues/93531