search

apache / beam

Apache Beam is a unified programming model for Batch and Streaming data processing.
https://beam.apache.org/
Apache License 2.0
7.88k stars 4.26k forks source link

Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet101-mean_inference_batch_latency_micro_secs:mean_inference_batch_latency_micro_secs #28067

Closed github-actions[bot] closed 1 year ago

github-actions[bot] commented 1 year ago

Performance change found in the test: pytorch_image_classification_benchmarks-resnet101-mean_inference_batch_latency_micro_secs for the metric: mean_inference_batch_latency_micro_secs.

For more information on how to triage the alerts, please look at Triage performance alert issues section of the README.

Test description: Pytorch image classification on 50k images of size 224 x 224 with resnet 101. Test link - https://github.com/apache/beam/blob/42d0a6e3564d8b9c5d912428a6de18fb22a13ac1/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy#L34 Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1&viewPanel=2


timestamp: Sat Aug 19 18:18:09 2023, metric_value: 4442828.55
timestamp: Fri Aug 18 18:18:58 2023, metric_value: 4518560.76
timestamp: Thu Aug 17 18:18:38 2023, metric_value: 4518552.26
timestamp: Wed Aug 16 18:18:41 2023, metric_value: 4572899.04
timestamp: Tue Aug 15 18:20:19 2023, metric_value: 4434901.51
timestamp: Mon Aug 14 18:19:03 2023, metric_value: 4499253.64
timestamp: Sun Aug 13 18:18:21 2023, metric_value: 4579284.80
timestamp: Sat Aug 12 18:18:01 2023, metric_value: 4540231.68
timestamp: Fri Aug 11 18:22:50 2023, metric_value: 4562882.83 <---- Anomaly
timestamp: Thu Aug 10 18:31:22 2023, metric_value: 4706314.64
timestamp: Wed Aug  9 18:20:39 2023, metric_value: 4499391.44
timestamp: Tue Aug  8 19:33:41 2023, metric_value: 4648045.91
timestamp: Mon Aug  7 18:21:00 2023, metric_value: 4527919.92
timestamp: Sun Aug  6 18:18:21 2023, metric_value: 4593467.59
timestamp: Sat Aug  5 18:18:37 2023, metric_value: 4640704.50
timestamp: Fri Aug  4 18:18:47 2023, metric_value: 4701437.57
timestamp: Thu Aug  3 18:30:50 2023, metric_value: 4478775.48
timestamp: Wed Aug  2 18:22:25 2023, metric_value: 4627538.10
timestamp: Wed Aug  2 13:43:54 2023, metric_value: 4730867.34
AnandInguva commented 1 year ago

Closing this as noise.