V2 Performance Signal Detected by TorchBench CI on '1.14.0.dev20221014+cu116'

[github-actions[bot]]

TorchBench CI has detected a performance signal.

Base PyTorch version: 1.14.0.dev20221013+cu116

Base PyTorch commit: 48c648d75df4a2d02ede71f34c11b7f48c80da0e

Affected PyTorch version: 1.14.0.dev20221014+cu116

Affected PyTorch commit: f451e824f39516f503c2bdfd785d254b447b9557

Affected Tests:

• test_train[Background_Matting-cuda-eager]: +17.71479%
• test_train[LearningToPaint-cpu-eager]: +9.90117%
• test_train[Super_SloMo-cuda-jit]: -26.05166%
• test_eval[Super_SloMo-cuda-jit]: -25.71798%
• test_train[Super_SloMo-cuda-eager]: -25.99412%
• test_eval[Super_SloMo-cuda-eager]: -25.67970%
• test_eval[demucs-cpu-jit]: +9.63566%
• test_eval[demucs-cpu-eager]: +8.87489%
• test_eval[drq-cuda-eager]: -12.05500%
• test_train[maml-cuda-eager]: -26.35052%
• test_eval[maml-cuda-eager]: -27.63354%
• test_train[maml_omniglot-cuda-eager]: -8.74326%
• test_eval[maml_omniglot-cuda-eager]: -12.38367%
• test_eval[opacus_cifar10-cuda-eager]: -12.02980%
• test_train[pytorch_stargan-cpu-jit]: +7.36235%
• test_eval[pytorch_stargan-cpu-jit]: +18.20981%
• test_eval[pytorch_stargan-cpu-eager]: +16.06476%
• test_train[pytorch_unet-cuda-jit]: -38.85917%
• test_eval[pytorch_unet-cuda-jit]: -24.33461%
• test_train[pytorch_unet-cuda-eager]: -38.86608%
• test_eval[pytorch_unet-cuda-eager]: -23.09795%
• test_train[timm_vision_transformer-cpu-eager]: -18.98734%
• test_train[yolov3-cuda-eager]: -31.50349%
• test_eval[yolov3-cuda-eager]: -34.86019%

cc @xuzhao9

[xuzhao9]

{
  "start": "48c648d75df4a2d02ede71f34c11b7f48c80da0e",
  "end": "f451e824f39516f503c2bdfd785d254b447b9557",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "b0d80f4355a",
      "commit1_time": "2022-10-13 17:20:35 +0000",
      "commit1_digest": {
        "test_train[Super_SloMo-cuda-jit]": 1.6595051454380154,
        "test_eval[Super_SloMo-cuda-jit]": 0.996937290718779,
        "test_train[Super_SloMo-cuda-eager]": 1.6624726527486928,
        "test_eval[Super_SloMo-cuda-eager]": 0.9987076044082641,
        "test_eval[drq-cuda-eager]": 0.002537990520502646,
        "test_train[maml-cuda-eager]": 0.8888606244116091,
        "test_eval[maml-cuda-eager]": 0.04880616907030344,
        "test_train[maml_omniglot-cuda-eager]": 5.442846909479703,
        "test_eval[maml_omniglot-cuda-eager]": 0.0011802791495238824,
        "test_eval[opacus_cifar10-cuda-eager]": 0.006044829535213384,
        "test_train[pytorch_unet-cuda-jit]": 0.80790025688475,
        "test_eval[pytorch_unet-cuda-jit]": 0.2062062485376373,
        "test_train[pytorch_unet-cuda-eager]": 0.8077503491309471,
        "test_eval[pytorch_unet-cuda-eager]": 0.21789157786406577,
        "test_train[yolov3-cuda-eager]": 1.3823745090165176,
        "test_eval[yolov3-cuda-eager]": 0.3579166673589498
      },
      "commit2": "427e0a6b4eb",
      "commit2_time": "2022-10-13 17:26:36 +0000",
      "commit2_digest": {
        "test_train[Super_SloMo-cuda-jit]": 1.2278542711748741,
        "test_eval[Super_SloMo-cuda-jit]": 0.7454392591724173,
        "test_train[Super_SloMo-cuda-eager]": 1.231106961099431,
        "test_eval[Super_SloMo-cuda-eager]": 0.7469768969458528,
        "test_eval[drq-cuda-eager]": 0.0022514490002303773,
        "test_train[maml-cuda-eager]": 0.653676811081823,
        "test_eval[maml-cuda-eager]": 0.035335327434385645,
        "test_train[maml_omniglot-cuda-eager]": 4.893214167561382,
        "test_eval[maml_omniglot-cuda-eager]": 0.0010374589926106118,
        "test_eval[opacus_cifar10-cuda-eager]": 0.005333516740533996,
        "test_train[pytorch_unet-cuda-jit]": 0.49549140566959976,
        "test_eval[pytorch_unet-cuda-jit]": 0.15605185518506914,
        "test_train[pytorch_unet-cuda-eager]": 0.49685618877410886,
        "test_eval[pytorch_unet-cuda-eager]": 0.16764753098832444,
        "test_train[yolov3-cuda-eager]": 0.9501905984827317,
        "test_eval[yolov3-cuda-eager]": 0.23672171049984173
      }
    }
  ]
}

[xuzhao9]

https://github.com/pytorch/pytorch/commit/427e0a6b4eb shows 10%-20% speedups across various CUDA workloads. Nice work! @malfet @eqy

[malfet]

Alas, the change has been reverted , but hopefully it could be relanded soon.