Performance Signal Detected by TorchBench CI on '1.11.0.dev20211023+cu102' - Githubissues

pytorch / benchmark

TorchBench is a collection of open source benchmarks used to evaluate PyTorch performance.

BSD 3-Clause "New" or "Revised" License

880 stars 288 forks source link

Performance Signal Detected by TorchBench CI on '1.11.0.dev20211023+cu102' #517

Closed github-actions[bot] closed 3 years ago

github-actions[bot] commented 3 years ago

TorchBench CI has detected a performance signal.

Base PyTorch version: 1.11.0.dev20211022+cu102

Base PyTorch commit: 5c9ba2feff08b98febd574514d79e2c8b6ffd703

Affected PyTorch version: 1.11.0.dev20211023+cu102

Affected PyTorch commit: db84da55b7c215761a009bc09a318270d33b0f7c

Affected Tests:

test_train[fastNLP-cpu-jit]: -7.86499%
test_eval[hf_Albert-cpu-eager]: -9.25840%
test_eval[hf_Bart-cpu-eager]: +8.20313%
test_eval[hf_GPT2-cpu-eager]: -12.19796%
test_eval[mobilenet_v2-cpu-eager]: -11.28131%
test_eval[pytorch_stargan-cpu-eager]: +13.90842%
test_train[soft_actor_critic-cpu-eager]: -17.53313%
test_eval[timm_vision_transformer-cpu-jit]: +20.28398%
test_eval[vgg16-cpu-eager]: +14.86526%

cc @xuzhao9

xuzhao9 commented 3 years ago

Closed for test purpose