V2 Performance Signal Detected by TorchBench CI on '2.0.0.dev20221215+cu116'

TorchBench CI has detected a performance signal.

Base PyTorch version: 2.0.0.dev20221214+cu116

Base PyTorch commit: 769392178a8da6aab1bf82a09c9b105ea3dd6778

Affected PyTorch version: 2.0.0.dev20221215+cu116

Affected PyTorch commit: 670efb974af056fc3f46b48225581351b047eb4d

Affected Tests:

test_train[Super_SloMo-cuda-jit]: -27.71048%
test_eval[Super_SloMo-cuda-jit]: -36.85259%
test_train[Super_SloMo-cuda-eager]: -27.70332%
test_eval[Super_SloMo-cuda-eager]: -36.82176%
test_eval[drq-cuda-eager]: -9.93525%
test_train[maml-cuda-eager]: -30.00386%
test_eval[maml-cuda-eager]: -30.96863%
test_train[maml_omniglot-cuda-eager]: -11.15875%
test_eval[maml_omniglot-cuda-eager]: -16.38785%
test_eval[opacus_cifar10-cuda-eager]: -12.05067%
test_train[pytorch_unet-cuda-jit]: -39.18751%
test_eval[pytorch_unet-cuda-jit]: -24.40643%
test_train[pytorch_unet-cuda-eager]: -39.18637%
test_eval[pytorch_unet-cuda-eager]: -23.13307%
test_train[timm_nfnet-cpu-jit]: -7.53174%
test_train[timm_vision_transformer-cpu-eager]: -7.89334%
test_train[yolov3-cuda-eager]: -37.16786%
test_eval[yolov3-cuda-eager]: -34.41439%

cc @xuzhao9

Result json:

{
  "start": "769392178a8da6aab1bf82a09c9b105ea3dd6778",
  "end": "670efb974af056fc3f46b48225581351b047eb4d",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "6a866c3ed18",
      "commit1_time": "2022-12-15 03:03:07 +0000",
      "commit1_digest": {
        "test_train[Super_SloMo-cuda-jit]": 1.603575493048993,
        "test_eval[Super_SloMo-cuda-jit]": 0.98791388105019,
        "test_train[Super_SloMo-cuda-eager]": 1.606425105803646,
        "test_eval[Super_SloMo-cuda-eager]": 0.9891651006444591,
        "test_eval[drq-cuda-eager]": 0.002439040604652142,
        "test_train[maml-cuda-eager]": 0.9507450418517692,
        "test_eval[maml-cuda-eager]": 0.053653370300889944,
        "test_train[maml_omniglot-cuda-eager]": 5.682578361401101,
        "test_eval[maml_omniglot-cuda-eager]": 0.0012058477448701366,
        "test_eval[opacus_cifar10-cuda-eager]": 0.006252907882093118,
        "test_train[pytorch_unet-cuda-jit]": 0.8034164728043833,
        "test_eval[pytorch_unet-cuda-jit]": 0.20602184999443124,
        "test_train[pytorch_unet-cuda-eager]": 0.8032078282965813,
        "test_eval[pytorch_unet-cuda-eager]": 0.21756679759419056,
        "test_train[yolov3-cuda-eager]": 1.424401665298501,
        "test_eval[yolov3-cuda-eager]": 0.35735864099988246
      },
      "commit2": "eecd621f06d",
      "commit2_time": "2022-12-15 03:24:44 +0000",
      "commit2_digest": {
        "test_train[Super_SloMo-cuda-jit]": 1.1589278396015288,
        "test_eval[Super_SloMo-cuda-jit]": 0.6243707725428976,
        "test_train[Super_SloMo-cuda-eager]": 1.161406543853809,
        "test_eval[Super_SloMo-cuda-eager]": 0.6252150069049094,
        "test_eval[drq-cuda-eager]": 0.002201703442355014,
        "test_train[maml-cuda-eager]": 0.6655023759958567,
        "test_eval[maml-cuda-eager]": 0.037237009486287005,
        "test_train[maml_omniglot-cuda-eager]": 5.070205841251299,
        "test_eval[maml_omniglot-cuda-eager]": 0.0010225446469401224,
        "test_eval[opacus_cifar10-cuda-eager]": 0.005485168019852911,
        "test_train[pytorch_unet-cuda-jit]": 0.48819210634101184,
        "test_eval[pytorch_unet-cuda-jit]": 0.15587643779581412,
        "test_train[pytorch_unet-cuda-eager]": 0.4884367838560138,
        "test_eval[pytorch_unet-cuda-eager]": 0.167215778096579,
        "test_train[yolov3-cuda-eager]": 0.8948697452520719,
        "test_eval[yolov3-cuda-eager]": 0.2342423016554676
      }
    }
  ]
}

Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/3707183290

pytorch / benchmark

V2 Performance Signal Detected by TorchBench CI on '2.0.0.dev20221215+cu116' #1348