V2 Performance Signal Detected by TorchBench CI on '2.0.0.dev20221218+cu116'

TorchBench CI has detected a performance signal.

Base PyTorch version: 2.0.0.dev20221217+cu116

Base PyTorch commit: e689c509222a2dbc4796bcf32625193b9ea13c49

Affected PyTorch version: 2.0.0.dev20221218+cu116

Affected PyTorch commit: e2377c8300c578d648aac3bff6f96cdfce4308de

Affected Tests:

test_train[Super_SloMo-cuda-jit]: +38.40817%
test_eval[Super_SloMo-cuda-jit]: +58.13357%
test_train[Super_SloMo-cuda-eager]: +38.29525%
test_eval[Super_SloMo-cuda-eager]: +57.74872%
test_eval[drq-cuda-eager]: +10.87170%
test_train[maml-cuda-eager]: +44.33552%
test_eval[maml-cuda-eager]: +46.23650%
test_train[maml_omniglot-cuda-eager]: +12.35584%
test_eval[maml_omniglot-cuda-eager]: +19.07364%
test_eval[mnasnet1_0-cpu-jit]: +51.14014%
test_eval[opacus_cifar10-cuda-eager]: +13.49595%
test_train[pytorch_unet-cuda-jit]: +64.62687%
test_eval[pytorch_unet-cuda-jit]: +32.26793%
test_train[pytorch_unet-cuda-eager]: +64.61674%
test_eval[pytorch_unet-cuda-eager]: +30.30729%
test_train[tts_angular-cpu-eager]: +10.25762%
test_train[yolov3-cuda-eager]: +58.84091%
test_eval[yolov3-cuda-eager]: +52.47351%

cc @xuzhao9

Result json:

{
  "start": "e689c509222a2dbc4796bcf32625193b9ea13c49",
  "end": "e2377c8300c578d648aac3bff6f96cdfce4308de",
  "threshold": 7,
  "timeout": 120,
  "torchbench_branch": "v2.0",
  "result": [
    {
      "commit1": "8bd959e462f",
      "commit1_time": "2022-12-17 00:30:06 +0000",
      "commit1_digest": {
        "test_train[Super_SloMo-cuda-jit]": 1.6034564033034258,
        "test_eval[Super_SloMo-cuda-jit]": 0.9874629732046742,
        "test_train[Super_SloMo-cuda-eager]": 1.6060320543940179,
        "test_eval[Super_SloMo-cuda-eager]": 0.9887237045506481,
        "test_eval[drq-cuda-eager]": 0.002446312502340822,
        "test_train[maml-cuda-eager]": 0.9542500565468799,
        "test_eval[maml-cuda-eager]": 0.053933588054496794,
        "test_train[maml_omniglot-cuda-eager]": 5.679953024134738,
        "test_eval[maml_omniglot-cuda-eager]": 0.0012070904142513477,
        "test_eval[mnasnet1_0-cpu-jit]": 0.0429693496262189,
        "test_eval[opacus_cifar10-cuda-eager]": 0.006260492848377907,
        "test_train[pytorch_unet-cuda-jit]": 0.8032183349394473,
        "test_eval[pytorch_unet-cuda-jit]": 0.2059999906574376,
        "test_train[pytorch_unet-cuda-eager]": 0.8031938458967488,
        "test_eval[pytorch_unet-cuda-eager]": 0.21753867400693708,
        "test_train[yolov3-cuda-eager]": 1.423063560394803,
        "test_eval[yolov3-cuda-eager]": 0.35733142900280657
      },
      "commit2": "ffa37c9fca6",
      "commit2_time": "2022-12-17 00:43:43 +0000",
      "commit2_digest": {
        "test_eval[mnasnet1_0-cpu-jit]": 0.036213897321180535
      }
    },
    {
      "commit1": "4438b019a8c",
      "commit1_time": "2022-12-16 22:34:33 +0000",
      "commit1_digest": {
        "test_train[Super_SloMo-cuda-jit]": 1.1580268270539817,
        "test_eval[Super_SloMo-cuda-jit]": 0.6236725504044444,
        "test_train[Super_SloMo-cuda-eager]": 1.1607889701990644,
        "test_eval[Super_SloMo-cuda-eager]": 0.6246151705010561,
        "test_eval[drq-cuda-eager]": 0.00220407683900914,
        "test_train[maml-cuda-eager]": 0.6614222453965339,
        "test_eval[maml-cuda-eager]": 0.03717009629838652,
        "test_train[maml_omniglot-cuda-eager]": 5.065122562897159,
        "test_eval[maml_omniglot-cuda-eager]": 0.0010215805991169054,
        "test_eval[opacus_cifar10-cuda-eager]": 0.005487575693871538,
        "test_train[pytorch_unet-cuda-jit]": 0.4882856906973757,
        "test_eval[pytorch_unet-cuda-jit]": 0.1557773525477387,
        "test_train[pytorch_unet-cuda-eager]": 0.4881072423973819,
        "test_eval[pytorch_unet-cuda-eager]": 0.16721102814772165,
        "test_train[yolov3-cuda-eager]": 0.8949597701430321,
        "test_eval[yolov3-cuda-eager]": 0.23422313841001596
      },
      "commit2": "ba7aeac37be",
      "commit2_time": "2022-12-16 23:06:35 +0000",
      "commit2_digest": {
        "test_train[Super_SloMo-cuda-jit]": 1.6032237276551313,
        "test_eval[Super_SloMo-cuda-jit]": 0.9872271865955554,
        "test_train[Super_SloMo-cuda-eager]": 1.605771068308968,
        "test_eval[Super_SloMo-cuda-eager]": 0.9882867571606766,
        "test_eval[drq-cuda-eager]": 0.0024396068128819575,
        "test_train[maml-cuda-eager]": 0.9526174238882958,
        "test_eval[maml-cuda-eager]": 0.05415626674657688,
        "test_train[maml_omniglot-cuda-eager]": 5.7255697060551025,
        "test_eval[maml_omniglot-cuda-eager]": 0.0012064494983691917,
        "test_eval[opacus_cifar10-cuda-eager]": 0.006257504650420742,
        "test_train[pytorch_unet-cuda-jit]": 0.803349940048065,
        "test_eval[pytorch_unet-cuda-jit]": 0.20602552760974505,
        "test_train[pytorch_unet-cuda-eager]": 0.8032204660528806,
        "test_eval[pytorch_unet-cuda-eager]": 0.21760045079281554,
        "test_train[yolov3-cuda-eager]": 1.427182837598957,
        "test_eval[yolov3-cuda-eager]": 0.35735670775175093
      }
    }
  ]
}

Bisection workflow link: https://github.com/pytorch/benchmark/actions/runs/3725850873

pytorch / benchmark

V2 Performance Signal Detected by TorchBench CI on '2.0.0.dev20221218+cu116' #1350