search

pytorch / benchmark

TorchBench is a collection of open source benchmarks used to evaluate PyTorch performance.
BSD 3-Clause "New" or "Revised" License
817 stars 259 forks source link

Fix test_bench script #2320

Closed xuzhao9 closed 1 week ago

xuzhao9 commented 1 week ago

Fixes https://github.com/pytorch/benchmark/issues/2315

Test Plan:

$ pytest test_bench.py -k "test_eval[BERT_pytorch-cpu]" --ignore_machine_config
========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.11.5, pytest-7.4.3, pluggy-1.0.0
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/xz/git/benchmark
plugins: benchmark-4.0.0, hypothesis-6.98.15
collected 411 items / 410 deselected / 1 selected

test_bench.py .                                                                                                                                                                                    [100%]

------------------------------------------------- benchmark 'hub': 1 tests -------------------------------------------------
Name (time in ms)                    Min       Max      Mean  StdDev    Median     IQR  Outliers     OPS  Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------
test_eval[BERT_pytorch-cpu]     114.2104  117.3853  115.4276  1.0485  115.3054  1.4325       4;0  8.6634       9           1
----------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
=================================================================================== 1 passed, 410 deselected in 5.68s ====================================================================================
facebook-github-bot commented 1 week ago

@xuzhao9 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 week ago

@xuzhao9 merged this pull request in pytorch/benchmark@8451c1f8fae21d78d21f5f02595fe15ce7c28fb0.