V2 Performance Signal Detected by TorchBench CI on '61239df555df02e8c60a2ad63363878a2a57c161'

TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 61239df555df02e8c60a2ad63363878a2a57c161

Affected PyTorch commit: 61239df555df02e8c60a2ad63363878a2a57c161

Affected Tests:

hf_T5_large, Adadelta, cuda, differentiable: +13.60454%
hf_T5_large, AdamW, cuda, no_foreach: +22.47358%
hf_T5_large, RAdam, cuda, (pt2) foreach: -11.14815%
hf_T5_large, RMSprop, cuda, no_foreach: +38.66255%
speech_transformer, Adagrad, cuda, default: -18.22282%
speech_transformer, Adam, cuda, (pt2) foreach, maximize, capturable: -20.42337%
speech_transformer, AdamW, cuda, foreach, maximize, capturable: +21.70409%
speech_transformer, ASGD, cuda, (pt2) default: +26.28217%
speech_transformer, ASGD, cuda, (pt2) foreach: +18.62524%
speech_transformer, Rprop, cuda, (pt2) foreach: +33.17368%
resnet152, Adagrad, cuda, (pt2) default: -16.15718%
resnet152, ASGD, cuda, (pt2) maximize: +17.12808%
resnet152, ASGD, cuda, (pt2) foreach: -12.05825%
resnet152, RAdam, cuda, (pt2) default: -19.17358%
resnet152, Rprop, cuda, (pt2) maximize: +17.61649%
resnet152, NAdam, cuda, (pt2) default: -14.12475%
resnet152, NAdam, cuda, default: -14.79072%
pytorch_struct, Adagrad, cuda, default: -10.95862%
pytorch_struct, Adagrad, cuda, maximize: -17.71966%
pytorch_struct, AdamW, cuda, (pt2) amsgrad, maximize: +13.53300%
pytorch_struct, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +23.93497%
pytorch_struct, AdamW, cuda, foreach, maximize, capturable, amsgrad: +14.15635%
pytorch_struct, Rprop, cuda, (pt2) default: -30.78712%
pytorch_struct, Rprop, cuda, default: -22.48682%
LearningToPaint, Adagrad, cuda, maximize: +16.84442%
LearningToPaint, Adam, cuda, foreach, maximize, capturable, amsgrad: -11.67310%
LearningToPaint, AdamW, cuda, amsgrad, maximize: +10.30468%
LearningToPaint, AdamW, cuda, foreach: +14.97183%
LearningToPaint, NAdam, cuda, (pt2) foreach: +10.06510%
timm_resnest, Adagrad, cuda, (pt2) maximize: -15.13953%
timm_resnest, Adagrad, cuda, foreach: +11.73446%
timm_resnest, Adam, cuda, foreach, maximize, capturable: -13.74881%
timm_resnest, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -13.23167%
timm_resnest, Rprop, cuda, maximize: +21.74018%
timm_nfnet, Adagrad, cuda, foreach: -10.77845%
timm_nfnet, AdamW, cuda, (pt2) foreach, maximize, capturable: +13.12893%
timm_nfnet, ASGD, cuda, (pt2) default: -19.26312%
timm_nfnet, ASGD, cuda, default: -17.34360%
timm_nfnet, ASGD, cuda, maximize: -14.62167%
densenet121, Adagrad, cuda, foreach: +22.43232%
densenet121, Adamax, cuda, maximize: -22.54708%
densenet121, Rprop, cuda, maximize: +29.38693%
densenet121, Rprop, cuda, foreach: -34.95967%
torchrec_dlrm, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -14.45357%
torchrec_dlrm, SGD, cuda, (pt2) foreach, momentum=0.9: -12.03607%
hf_T5, Adagrad, cuda, (pt2) maximize: +12.49735%
hf_T5, Adagrad, cuda, foreach: -31.39563%
hf_T5, Adam, cuda, foreach, maximize, capturable: +17.29529%
hf_T5, Adamax, cuda, (pt2) default: +31.25084%
hf_T5, Rprop, cuda, maximize: +15.05087%
detectron2_fasterrcnn_r_101_fpn, Adamax, cuda, (pt2) maximize: -13.24207%
fastNLP_Bert, AdamW, cuda, foreach, maximize, capturable: -11.34146%
Background_Matting, Adagrad, cuda, (pt2) maximize: +15.44612%
Background_Matting, AdamW, cuda, (pt2) default: +14.85950%
doctr_det_predictor, Adagrad, cuda, maximize: -28.51256%
doctr_det_predictor, AdamW, cuda, foreach: -11.91730%
doctr_det_predictor, AdamW, cuda, (pt2) foreach, maximize, capturable: +11.17356%
doctr_det_predictor, AdamW, cuda, foreach, maximize, capturable: +22.57804%
doctr_det_predictor, ASGD, cuda, (pt2) default: +13.07918%
doctr_det_predictor, Rprop, cuda, maximize: -16.71013%
mnasnet1_0, Adagrad, cuda, default: +11.33607%
mnasnet1_0, Adam, cuda, default: +12.69658%
mnasnet1_0, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -19.85623%
mnasnet1_0, Adamax, cuda, (pt2) default: -25.29040%
mnasnet1_0, ASGD, cuda, (pt2) maximize: +15.28812%
mnasnet1_0, Rprop, cuda, (pt2) default: -12.66443%
detectron2_fasterrcnn_r_50_c4, Adamax, cuda, (pt2) maximize: -12.23157%
detectron2_fasterrcnn_r_50_c4, ASGD, cuda, (pt2) foreach: -15.23407%
detectron2_fasterrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -27.15889%
detectron2_fasterrcnn_r_50_c4, Rprop, cuda, (pt2) maximize: -30.22095%
detectron2_fasterrcnn_r_50_c4, Rprop, cuda, (pt2) foreach: +35.17799%
timm_vovnet, Adagrad, cuda, maximize: -30.08798%
timm_vovnet, Adamax, cuda, default: +30.26049%
timm_vovnet, Adamax, cuda, (pt2) maximize: -15.23673%
timm_vovnet, Adamax, cuda, maximize: +28.61749%
timm_vovnet, RAdam, cuda, default: +18.71114%
timm_vovnet, Rprop, cuda, (pt2) default: +22.53426%
timm_vovnet, Rprop, cuda, (pt2) foreach: +44.75472%
timm_vovnet, Rprop, cuda, foreach: +21.76370%
phlippe_resnet, Adam, cuda, (pt2) default: +14.10820%
phlippe_resnet, Adam, cuda, foreach, maximize, capturable, amsgrad: -12.74643%
phlippe_resnet, Adamax, cuda, (pt2) default: -26.51051%
phlippe_resnet, Adamax, cuda, (pt2) maximize: +32.45039%
phlippe_resnet, ASGD, cuda, maximize: -15.60158%
phlippe_resnet, SGD, cuda, (pt2) foreach, momentum=0.9: -10.03034%
phlippe_densenet, Adam, cuda, foreach: +10.65175%
phlippe_densenet, Adam, cuda, foreach, maximize, capturable: -10.98485%
phlippe_densenet, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +15.39777%
phlippe_densenet, Rprop, cuda, (pt2) maximize: +11.40993%
phlippe_densenet, Rprop, cuda, maximize: +13.60870%
squeezenet1_1, Adagrad, cuda, default: +14.66994%
squeezenet1_1, Adagrad, cuda, (pt2) foreach: -11.50462%
squeezenet1_1, Adam, cuda, (pt2) foreach, maximize, capturable: +16.25525%
squeezenet1_1, Adamax, cuda, (pt2) maximize: -25.69500%
squeezenet1_1, Adamax, cuda, foreach: +23.13726%
squeezenet1_1, RAdam, cuda, (pt2) default: +11.68151%
squeezenet1_1, Rprop, cuda, (pt2) maximize: +37.85334%
squeezenet1_1, Rprop, cuda, maximize: +10.00621%
squeezenet1_1, Rprop, cuda, foreach: -22.33141%
squeezenet1_1, NAdam, cuda, (pt2) foreach: +10.97497%
fambench_xlmr, ASGD, cuda, (pt2) default: -11.41906%
fambench_xlmr, Rprop, cuda, default: +10.29543%
mobilenet_v2_quantized_qat, Adagrad, cuda, default: -13.70634%
mobilenet_v2_quantized_qat, Adagrad, cuda, (pt2) foreach: +11.59934%
mobilenet_v2_quantized_qat, AdamW, cuda, default: +11.92230%
mobilenet_v2_quantized_qat, Rprop, cuda, maximize: +10.19087%
detectron2_fasterrcnn_r_50_fpn, Adagrad, cuda, (pt2) default: -26.19251%
detectron2_fasterrcnn_r_50_fpn, Adagrad, cuda, default: -17.72972%
detectron2_fasterrcnn_r_50_fpn, ASGD, cuda, default: -11.05838%
detectron2_fasterrcnn_r_50_fpn, Rprop, cuda, maximize: +18.29244%
timm_regnet, Adagrad, cuda, foreach: -31.48542%
timm_regnet, Adam, cuda, foreach, maximize, capturable: -11.32439%
timm_regnet, AdamW, cuda, (pt2) foreach, maximize, capturable: +14.77964%
timm_regnet, ASGD, cuda, maximize: +10.27234%
timm_regnet, ASGD, cuda, foreach: -25.83032%
timm_regnet, Rprop, cuda, default: +11.85468%
resnet50, Adadelta, cuda, (pt2) differentiable: -12.23494%
resnet50, Adagrad, cuda, (pt2) no_foreach: +10.04018%
resnet50, Adagrad, cuda, differentiable: +47.46322%
resnet50, Adagrad, cuda, (pt2) foreach: +44.43562%
resnet50, Adam, cuda, (pt2) differentiable: +11.25322%
resnet50, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +10.26715%
resnet50, AdamW, cuda, default: -11.16829%
resnet50, Adamax, cuda, (pt2) default: +11.47901%
resnet50, Adamax, cuda, (pt2) maximize: -16.38561%
resnet50, Adamax, cuda, (pt2) foreach: -14.60888%
resnet50, ASGD, cuda, (pt2) maximize: +28.51260%
resnet50, RAdam, cuda, (pt2) differentiable: -15.08969%
resnet50, RAdam, cuda, (pt2) foreach: -16.58168%
resnet50, Rprop, cuda, foreach: +27.03858%
resnet50, NAdam, cuda, (pt2) default: +10.26112%
hf_Bert, Adagrad, cuda, (pt2) foreach: +19.31554%
hf_Bert, Adamax, cuda, default: +23.80594%
lennard_jones, Adadelta, cuda, maximize: -10.65196%
lennard_jones, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -12.54647%
lennard_jones, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +10.95844%
lennard_jones, ASGD, cuda, (pt2) default: +29.08472%
lennard_jones, ASGD, cuda, maximize: +35.90132%
lennard_jones, ASGD, cuda, (pt2) foreach: -19.84921%
lennard_jones, ASGD, cuda, foreach: -14.93836%
lennard_jones, SGD, cuda, foreach, momentum=0.9: +14.93877%
lennard_jones, RAdam, cuda, default: -11.16003%
lennard_jones, Rprop, cuda, (pt2) maximize: +10.55758%
lennard_jones, Rprop, cuda, foreach: -17.00680%
lennard_jones, RMSprop, cuda, foreach: +10.84504%
timm_efficientnet, Adagrad, cuda, (pt2) default: +17.82986%
timm_efficientnet, Adagrad, cuda, (pt2) maximize: +14.37078%
timm_efficientdet, Adagrad, cuda, maximize: +11.55644%
timm_efficientdet, Adam, cuda, amsgrad, maximize: -12.94190%
timm_efficientdet, Adamax, cuda, (pt2) default: -20.94779%
timm_efficientdet, Adamax, cuda, maximize: +16.95827%
timm_efficientdet, ASGD, cuda, (pt2) default: +29.36164%
timm_efficientdet, Rprop, cuda, (pt2) maximize: +10.00228%
timm_efficientdet, Rprop, cuda, (pt2) foreach: +13.73209%
timm_efficientdet, NAdam, cuda, default: +16.40506%
drq, Adagrad, cuda, default: -29.40278%
drq, AdamW, cuda, (pt2) foreach: +10.80673%
drq, Adamax, cuda, maximize: -23.21681%
drq, ASGD, cuda, maximize: -14.23941%
drq, RAdam, cuda, (pt2) default: +14.06625%
drq, RAdam, cuda, (pt2) foreach: -15.32356%
hf_T5_generate, Adagrad, cuda, (pt2) maximize: +13.75225%
hf_T5_generate, Adagrad, cuda, (pt2) foreach: -22.28028%
hf_T5_generate, Adamax, cuda, default: +14.27014%
hf_T5_generate, RAdam, cuda, (pt2) default: +11.47564%
hf_T5_generate, Rprop, cuda, maximize: -11.78328%
hf_T5_generate, Rprop, cuda, (pt2) foreach: -10.57932%
hf_T5_generate, Rprop, cuda, foreach: -32.73970%
detectron2_maskrcnn_r_50_c4, Adagrad, cuda, maximize: -18.85365%
moco, ASGD, cuda, (pt2) foreach: +29.08947%
moco, Rprop, cuda, (pt2) foreach: -18.29600%
basic_gnn_gin, Adagrad, cuda, default: -16.43837%
basic_gnn_gin, Adam, cuda, (pt2) default: -10.71454%
basic_gnn_gin, Rprop, cuda, maximize: -10.41602%
resnet50_quantized_qat, Adagrad, cuda, (pt2) maximize: -10.06705%
resnet50_quantized_qat, Adamax, cuda, foreach: -12.15751%
resnet50_quantized_qat, ASGD, cuda, default: +18.22748%
resnet50_quantized_qat, ASGD, cuda, maximize: +20.63210%
vision_maskrcnn, Adagrad, cuda, default: -13.73216%
vision_maskrcnn, Adam, cuda, (pt2) foreach, maximize, capturable: -12.29962%
vision_maskrcnn, Adamax, cuda, (pt2) foreach: -25.53220%
vision_maskrcnn, Adamax, cuda, foreach: +20.03636%
vision_maskrcnn, ASGD, cuda, (pt2) foreach: -17.80287%
vision_maskrcnn, ASGD, cuda, foreach: +26.06824%
vision_maskrcnn, Rprop, cuda, maximize: +48.69072%
vision_maskrcnn, Rprop, cuda, foreach: +48.50271%
hf_GPT2, Adagrad, cuda, default: -10.43195%
hf_GPT2, Rprop, cuda, (pt2) default: +40.37547%
hf_GPT2, Rprop, cuda, (pt2) maximize: -21.21464%
DALLE2_pytorch, Adagrad, cuda, (pt2) maximize: -28.31618%
DALLE2_pytorch, Adagrad, cuda, maximize: -29.68450%
DALLE2_pytorch, Adagrad, cuda, foreach: -13.56571%
DALLE2_pytorch, AdamW, cuda, fused, amsgrad, maximize: -29.81193%
DALLE2_pytorch, Rprop, cuda, default: +14.65963%
DALLE2_pytorch, Rprop, cuda, (pt2) foreach: -10.47113%
attention_is_all_you_need_pytorch, AdamW, cuda, (pt2) foreach, maximize, capturable: -13.58209%
attention_is_all_you_need_pytorch, ASGD, cuda, (pt2) foreach: -13.25113%
attention_is_all_you_need_pytorch, ASGD, cuda, foreach: +25.25339%
functorch_maml_omniglot, Adagrad, cuda, maximize: -27.84574%
functorch_maml_omniglot, AdamW, cuda, (pt2) default: -10.29754%
functorch_maml_omniglot, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -12.11789%
functorch_maml_omniglot, ASGD, cuda, default: -13.95462%
functorch_maml_omniglot, RAdam, cuda, foreach: -10.81749%
detectron2_maskrcnn, Adagrad, cuda, (pt2) default: +10.03808%
detectron2_maskrcnn, Adamax, cuda, (pt2) default: -11.67045%
detectron2_maskrcnn, ASGD, cuda, maximize: +25.72164%
detectron2_maskrcnn, Rprop, cuda, maximize: +30.01440%
basic_gnn_gcn, Adam, cuda, foreach: +10.09730%
basic_gnn_gcn, Adam, cuda, (pt2) foreach, maximize, capturable: +20.37854%
basic_gnn_gcn, Adam, cuda, foreach, maximize, capturable: +11.82814%
basic_gnn_gcn, Adamax, cuda, default: -24.98751%
basic_gnn_gcn, NAdam, cuda, foreach: +11.66020%
opacus_cifar10, Adam, cuda, foreach: -12.82273%
opacus_cifar10, Adamax, cuda, default: +21.08431%
opacus_cifar10, Adamax, cuda, (pt2) maximize: +32.02716%
opacus_cifar10, Adamax, cuda, foreach: +27.23378%
opacus_cifar10, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -28.65616%
opacus_cifar10, Rprop, cuda, (pt2) maximize: -23.25056%
hf_DistilBert, Adagrad, cuda, foreach: +10.68220%
hf_DistilBert, Adamax, cuda, (pt2) foreach: +12.05817%
hf_DistilBert, ASGD, cuda, (pt2) maximize: -13.33010%
hf_DistilBert, ASGD, cuda, maximize: -26.60854%
hf_DistilBert, Rprop, cuda, (pt2) foreach: -18.89753%
hf_DistilBert, Rprop, cuda, foreach: -10.80559%
functorch_dp_cifar10, Adagrad, cuda, (pt2) default: -28.36952%
functorch_dp_cifar10, Adagrad, cuda, default: -22.25236%
functorch_dp_cifar10, Adagrad, cuda, foreach: -18.63088%
functorch_dp_cifar10, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +24.07712%
maml_omniglot, Adadelta, cuda, maximize: +15.74202%
maml_omniglot, Adam, cuda, (pt2) amsgrad, maximize: -13.92704%
maml_omniglot, Adam, cuda, fused: -10.21255%
maml_omniglot, ASGD, cuda, (pt2) default: -26.65473%
maml_omniglot, ASGD, cuda, default: -20.93049%
maml_omniglot, NAdam, cuda, foreach: -15.05091%
mobilenet_v3_large, Adagrad, cuda, (pt2) foreach: -26.48557%
mobilenet_v3_large, AdamW, cuda, foreach, maximize, capturable: +10.80723%
mobilenet_v3_large, Adamax, cuda, default: -12.13923%
mobilenet_v3_large, Adamax, cuda, (pt2) foreach: +29.29962%
mobilenet_v3_large, ASGD, cuda, (pt2) default: -18.29511%
mobilenet_v3_large, ASGD, cuda, default: +14.85064%
mobilenet_v3_large, ASGD, cuda, foreach: -29.75856%
mobilenet_v3_large, RAdam, cuda, default: -10.62867%
nanogpt_generate, Rprop, cuda, (pt2) default: +41.21039%
nanogpt_generate, Rprop, cuda, (pt2) maximize: +41.27364%
detectron2_maskrcnn_r_101_c4, Adamax, cuda, foreach: -26.89252%
detectron2_maskrcnn_r_101_c4, ASGD, cuda, (pt2) default: -12.44815%
detectron2_maskrcnn_r_101_c4, Rprop, cuda, default: +11.11972%
detectron2_fasterrcnn_r_50_dc5, SGD, cuda, (pt2) foreach, momentum=0.9: +20.00735%
hf_Longformer, Adamax, cuda, default: +15.61983%
hf_Longformer, Rprop, cuda, (pt2) default: +41.45902%
basic_gnn_edgecnn, Rprop, cuda, (pt2) default: -13.05964%
hf_Albert, ASGD, cuda, default: -14.27121%
hf_Albert, ASGD, cuda, (pt2) foreach: -19.49115%
yolov3, Adagrad, cuda, (pt2) no_foreach: +16.03149%
yolov3, Adagrad, cuda, foreach: +18.63486%
yolov3, Adam, cuda, no_foreach: +14.38061%
yolov3, Adam, cuda, differentiable: -16.54344%
yolov3, AdamW, cuda, (pt2) differentiable: +10.71704%
yolov3, AdamW, cuda, differentiable: +37.27744%
yolov3, AdamW, cuda, foreach, maximize, capturable, amsgrad: +21.15375%
yolov3, Adamax, cuda, differentiable: +12.35434%
yolov3, ASGD, cuda, (pt2) differentiable: +10.44449%
yolov3, ASGD, cuda, (pt2) foreach: +49.38943%
yolov3, ASGD, cuda, foreach: +32.95941%
yolov3, RAdam, cuda, (pt2) default: -12.58476%
yolov3, RAdam, cuda, (pt2) no_foreach: -11.84660%
yolov3, RAdam, cuda, no_foreach: +32.02452%
yolov3, RAdam, cuda, foreach: +22.91353%
yolov3, Rprop, cuda, no_foreach: -30.77830%
yolov3, Rprop, cuda, (pt2) differentiable: -10.01303%
yolov3, Rprop, cuda, (pt2) foreach: +14.83490%
yolov3, Rprop, cuda, foreach: +15.79699%
soft_actor_critic, AdamW, cuda, default: -17.33461%
soft_actor_critic, AdamW, cuda, (pt2) foreach, maximize, capturable: +12.61778%
soft_actor_critic, Adamax, cuda, (pt2) default: +22.45962%
soft_actor_critic, Adamax, cuda, default: +14.58660%
soft_actor_critic, RMSprop, cuda, foreach: +11.66893%
soft_actor_critic, NAdam, cuda, (pt2) default: +13.45134%
detectron2_maskrcnn_r_50_fpn, AdamW, cuda, (pt2) foreach, maximize, capturable: +10.67271%
detectron2_maskrcnn_r_50_fpn, AdamW, cuda, foreach, maximize, capturable: +14.53275%
detectron2_maskrcnn_r_50_fpn, ASGD, cuda, (pt2) default: -27.12756%
detectron2_maskrcnn_r_50_fpn, Rprop, cuda, (pt2) default: -31.44503%
resnet18, Adam, cuda, (pt2) foreach: +13.00038%
resnet18, Adam, cuda, foreach: -12.64608%
resnet18, ASGD, cuda, (pt2) maximize: -12.30010%
resnet18, RAdam, cuda, (pt2) foreach: +19.26040%
resnext50_32x4d, Adagrad, cuda, maximize: -10.48879%
resnext50_32x4d, RAdam, cuda, foreach: +13.25582%
mobilenet_v2, Adagrad, cuda, maximize: -26.17869%
mobilenet_v2, Adagrad, cuda, foreach: -10.65649%
mobilenet_v2, Adam, cuda, (pt2) default: +10.97607%
mobilenet_v2, ASGD, cuda, (pt2) foreach: +51.72550%
mobilenet_v2, Rprop, cuda, (pt2) maximize: +10.77751%
mobilenet_v2, Rprop, cuda, foreach: -15.58003%
hf_Reformer, Adagrad, cuda, maximize: +11.16544%
hf_Reformer, ASGD, cuda, (pt2) foreach: -32.21486%
hf_Reformer, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +19.80901%
hf_Reformer, SGD, cuda, (pt2) foreach, momentum=0.9: -49.98759%
tts_angular, Adagrad, cuda, default: -11.57109%
tts_angular, AdamW, cuda, foreach, maximize, capturable: -21.02603%
tts_angular, Rprop, cuda, default: +18.57440%
tts_angular, Rprop, cuda, maximize: -10.53156%
tts_angular, Rprop, cuda, foreach: +43.81550%
hf_Bert_large, Adagrad, cuda, foreach: -12.47307%
hf_Bert_large, ASGD, cuda, default: +28.44674%
hf_Bert_large, Rprop, cuda, maximize: -11.82824%
hf_T5_base, Adagrad, cuda, maximize: -14.03934%
hf_T5_base, Adamax, cuda, (pt2) default: +21.21990%
hf_T5_base, Adamax, cuda, default: -20.78679%
hf_T5_base, ASGD, cuda, default: -23.59413%
timm_vision_transformer, Adadelta, cuda, (pt2) no_foreach: +11.41676%
timm_vision_transformer, Adadelta, cuda, no_foreach: +10.20763%
timm_vision_transformer, Adadelta, cuda, differentiable: -22.33240%
timm_vision_transformer, Adagrad, cuda, maximize: -17.42072%
timm_vision_transformer, AdamW, cuda, (pt2) foreach, maximize, capturable: +18.71245%
timm_vision_transformer, AdamW, cuda, foreach, maximize, capturable: -12.43316%
timm_vision_transformer, Adamax, cuda, (pt2) maximize: -10.24628%
timm_vision_transformer, Adamax, cuda, (pt2) no_foreach: +10.77915%
timm_vision_transformer, ASGD, cuda, default: -13.83146%
timm_vision_transformer, ASGD, cuda, (pt2) foreach: +13.61679%
timm_vision_transformer, SGD, cuda, no_foreach: +48.24445%
timm_vision_transformer, RAdam, cuda, no_foreach: +21.56532%
timm_vision_transformer, Rprop, cuda, (pt2) foreach: +46.90374%
hf_BigBird, ASGD, cuda, maximize: +16.82669%
hf_BigBird, Rprop, cuda, (pt2) default: +34.88466%
maml, Adam, cuda, (pt2) default: +14.16109%
maml, Adamax, cuda, (pt2) foreach: +33.64934%
maml, ASGD, cuda, maximize: -16.86106%
maml, ASGD, cuda, (pt2) foreach: +41.76913%
maml, RAdam, cuda, (pt2) default: -11.01540%
maml, Rprop, cuda, (pt2) maximize: +46.33812%
pytorch_stargan, Adam, cuda, amsgrad, maximize: -13.45159%
pytorch_stargan, AdamW, cuda, (pt2) foreach, maximize, capturable: -18.92852%
pytorch_stargan, AdamW, cuda, foreach, maximize, capturable: -11.42102%
pytorch_stargan, Adamax, cuda, (pt2) default: -25.06524%
pytorch_stargan, ASGD, cuda, maximize: -22.20560%
pytorch_stargan, Rprop, cuda, default: +16.42667%
doctr_reco_predictor, Adagrad, cuda, default: +11.81406%
doctr_reco_predictor, Adagrad, cuda, foreach: -13.42570%
doctr_reco_predictor, Adamax, cuda, (pt2) maximize: -21.20705%
doctr_reco_predictor, ASGD, cuda, (pt2) default: +12.56589%
doctr_reco_predictor, ASGD, cuda, maximize: +13.22545%
doctr_reco_predictor, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +11.98373%
doctr_reco_predictor, Rprop, cuda, (pt2) default: +30.54472%
hf_GPT2_large, Rprop, cuda, (pt2) foreach: +12.82935%
hf_GPT2_large, NAdam, cuda, default: -12.72739%
pytorch_CycleGAN_and_pix2pix, Adam, cuda, (pt2) foreach, maximize, capturable: -13.52605%
pytorch_CycleGAN_and_pix2pix, RAdam, cuda, (pt2) foreach: +11.45351%
pytorch_CycleGAN_and_pix2pix, RAdam, cuda, foreach: +27.26347%
pytorch_CycleGAN_and_pix2pix, Rprop, cuda, (pt2) maximize: +25.12776%
shufflenet_v2_x1_0, Adagrad, cuda, (pt2) foreach: +10.18825%
shufflenet_v2_x1_0, Rprop, cuda, (pt2) foreach: +13.46989%
BERT_pytorch, Adadelta, cuda, differentiable: -37.41114%
BERT_pytorch, Adagrad, cuda, foreach: +16.35415%
BERT_pytorch, AdamW, cuda, (pt2) differentiable: +11.26834%
BERT_pytorch, Adamax, cuda, no_foreach: -17.72422%
BERT_pytorch, Adamax, cuda, differentiable: +18.66413%
BERT_pytorch, RMSprop, cuda, no_foreach: +50.91613%
dcgan, Adagrad, cuda, (pt2) foreach: -22.44141%
dcgan, Adam, cuda, default: -11.44944%
dcgan, Adam, cuda, foreach, maximize, capturable: -11.43638%
dcgan, AdamW, cuda, (pt2) foreach: -12.85182%
dcgan, Adamax, cuda, foreach: -22.16777%
dcgan, ASGD, cuda, (pt2) default: +30.91377%
dcgan, ASGD, cuda, foreach: -20.35816%
dcgan, Rprop, cuda, maximize: -27.10498%
detectron2_maskrcnn_r_101_fpn, Adagrad, cuda, foreach: -17.31563%
detectron2_maskrcnn_r_101_fpn, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -11.97197%
detectron2_maskrcnn_r_101_fpn, Adamax, cuda, (pt2) default: -24.95229%
detectron2_maskrcnn_r_101_fpn, Adamax, cuda, maximize: +32.51212%
detectron2_maskrcnn_r_101_fpn, Adamax, cuda, (pt2) foreach: -14.26331%
detectron2_maskrcnn_r_101_fpn, Adamax, cuda, foreach: +31.28277%
detectron2_maskrcnn_r_101_fpn, ASGD, cuda, foreach: +14.06535%
Super_SloMo, Adamax, cuda, (pt2) foreach: +17.66255%
Super_SloMo, ASGD, cuda, (pt2) default: +11.44557%
Super_SloMo, ASGD, cuda, default: +20.54556%
tacotron2, Adagrad, cuda, (pt2) foreach: -24.03062%
tacotron2, ASGD, cuda, (pt2) foreach: -11.73387%
tacotron2, SGD, cuda, (pt2) foreach, momentum=0.9: -13.48581%
tacotron2, Rprop, cuda, default: -23.21057%
llama, Adam, cuda, amsgrad, maximize: -10.00944%
llama, Adamax, cuda, default: +19.25434%
llama, Rprop, cuda, maximize: +41.28893%
llama, NAdam, cuda, (pt2) default: +13.53627%
basic_gnn_sage, Adagrad, cuda, (pt2) default: +27.84624%
basic_gnn_sage, Adagrad, cuda, maximize: -13.68657%
basic_gnn_sage, Adam, cuda, foreach, maximize, capturable, amsgrad: -13.48494%
basic_gnn_sage, AdamW, cuda, (pt2) amsgrad, maximize: -10.66960%
basic_gnn_sage, SGD, cuda, foreach, momentum=0.9: +13.97110%
basic_gnn_sage, Rprop, cuda, (pt2) default: -11.87209%
basic_gnn_sage, Rprop, cuda, (pt2) foreach: -18.34679%
pytorch_unet, Adagrad, cuda, (pt2) foreach: +13.24542%
pytorch_unet, Adamax, cuda, default: -10.03720%
pytorch_unet, ASGD, cuda, foreach: +28.25222%
pytorch_unet, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -16.88537%
pytorch_unet, SGD, cuda, (pt2) foreach, momentum=0.9: -18.45504%
pytorch_unet, Rprop, cuda, (pt2) default: +47.62757%
detectron2_fasterrcnn_r_101_c4, Adagrad, cuda, (pt2) default: +34.30178%
detectron2_fasterrcnn_r_101_c4, Adam, cuda, foreach, maximize, capturable: +16.00049%
detectron2_fasterrcnn_r_101_c4, Adamax, cuda, (pt2) default: -19.76335%
detectron2_fasterrcnn_r_101_c4, Adamax, cuda, (pt2) maximize: +20.19919%
detectron2_fasterrcnn_r_101_c4, ASGD, cuda, default: +11.44166%
hf_Bart, ASGD, cuda, maximize: -11.23174%
hf_Bart, Rprop, cuda, (pt2) maximize: +12.43215%

Tests that were no longer run on affected commit:

Tests that were newly added on affected commit:

Runtime regressions found? No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/5035713765

cc @janeyx99

pytorch / benchmark

V2 Performance Signal Detected by TorchBench CI on '61239df555df02e8c60a2ad63363878a2a57c161' #1673