search

mongodb / mongo-perf

performance tools for mongodb
351 stars 140 forks source link

SERVER-90613 Zixuan Skunkworks 2024: Improve microbenchmark benchrun.py #158

Closed leozzx closed 5 months ago

leozzx commented 6 months ago
  1. Includes median and stdev for each test result
  2. Allows users to provide a variant name and list of variant values and comparing the perf with different variants.
  3. Allow users to use regex for filtering test names
  4. Prints a summary at the end of test.

https://spruce.mongodb.com/version/664bdf7ab3280b0007d8c421/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC

New --tsvSummary option output looks like


name                     variant    thread_count    ops_per_sec(mean)    ops_per_sec(median)    count    stdev
---------------------  ---------  --------------  -------------------  ---------------------  -------  -------
Aggregation.Group.All          0               1            1628.9812              1627.5134        3   3.7440
Aggregation.Group.All          1               1            1576.7357              1571.2285        3  17.9333
Aggregation.Group.All          2               1            1620.0208              1606.8993        3  20.7279
Aggregation.Group.All          4               1            1592.5159              1593.8476        3   5.9513
Aggregation.Group.All          8               1            1603.3958              1583.7102        3  29.6256
anna-wawrzyniak commented 5 months ago

The new json output is backwards compatible with the previous one, right?

leozzx commented 5 months ago

The new json output is backwards compatible with the previous one, right?

Yes, I just add some additional fields to it.