search

cloud-bulldozer / benchmark-comparison

Project to help compare benchmark result data and system metric data provided from benchmark run.
Other
1 stars 23 forks source link

Standarize benchmark-comparison to config files and implement tolerations #41

Closed rsevilla87 closed 3 years ago

rsevilla87 commented 3 years ago

Description

With this PR benchmark-comparison will rely on configuration files rather than hardcoded python code to perform queries. It provides the capability of reusing the current code for different benchmarks w/o needing to modify any python code but only the configuration.

Additional info can be found in the updated README.md.

i.e

$ touchstone_compare -url <hidden> -u a4afc882-101d-576d-a981-78c422d7aa97 eca3c4eb-46f4-5136-adc7-c1acedcffd2e --config config/uperf.json --tolerancy-rules tolerancy-configs/uperf.yaml                                                                                                                                                                                                                                                                                               

=================================================================== UUID: a4afc882-101d-576d-a981-78c422d7aa97 ===================================================================                                                                                                                                           
+-------------------------+------------------------------------------------+--------------------+--------------+                                                                                                                                                                                                             
|          where          |                value.Model name                | value.Architecture | value.CPU(s) |                                                                                                                                                                                                             
+-------------------------+------------------------------------------------+--------------------+--------------+                                                                                                                                                                                                             
| backpack-a4afc882-d4z5g | Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |       x86_64       |      4       |                                                                                                                                                                                                             
| backpack-a4afc882-trnhb | Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |       x86_64       |      8       |                                                                                                                                                                                                             
| backpack-a4afc882-f22zr | Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz  |       x86_64       |      8       |                                                                                                                                                                                                             
| backpack-a4afc882-cdmdd | Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |       x86_64       |      4       |                                                                                                                                                                                                             
| backpack-a4afc882-l82lr | Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |       x86_64       |      8       |                                                                                                                                                                                                             
| backpack-a4afc882-4pzkz | Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |       x86_64       |      4       |                                                                                                                                                                                                             
+-------------------------+------------------------------------------------+--------------------+--------------+                                                                                                                                                                                                             

=================================================================== UUID: a4afc882-101d-576d-a981-78c422d7aa97 ===================================================================                                                                                                                                           
+-------------------------+----------------+                                                                                                                                                                                                                                                                                 
|          where          | value.MemTotal |                                                                                                                                                                                                                                                                                 
+-------------------------+----------------+                                                                                                                                                                                                                                                                                 
| backpack-a4afc882-trnhb |  32458320 kB   |                                                                                                                                                                                                                                                                                 
| backpack-a4afc882-l82lr |  32458320 kB   |                                                                                                                                                                                                                                                                                 
| backpack-a4afc882-d4z5g |  16116152 kB   |                                                                                                                                                                                                                                                                                 
| backpack-a4afc882-cdmdd |  16116152 kB   |                                                                                                                                                                                                                                                                                 
| backpack-a4afc882-f22zr |  32114260 kB   |                                                                                                                                                                                                                                                                                 
| backpack-a4afc882-4pzkz |  16116152 kB   |                                                                                                                                                                                                                                                                                 
+-------------------------+----------------+                                                                                                                                                                                                                                                                                 

=================================================================== UUID: a4afc882-101d-576d-a981-78c422d7aa97 ===================================================================                                                                                                                                           
+-------+-----------------------+                                                                                                                                                                                                                                                                                            
| where | value.cluster_version |                                                                                                                                                                                                                                                                                            
+-------+-----------------------+                                                                                                                                                                                                                                                                                            
|  NA   |        4.6.13         |                                                                                                                                                                                                                                                                                            
+-------+-----------------------+                                                                                                                                                                                                                                                                                            

=================================================================== UUID: eca3c4eb-46f4-5136-adc7-c1acedcffd2e ===================================================================                                                                                                                                           
+----------------------------------------+------------------------------------------------+--------------------+--------------+                                                                                                                                                                                              
|                 where                  |                value.Model name                | value.Architecture | value.CPU(s) |                                                                                                                                                                                              
+----------------------------------------+------------------------------------------------+--------------------+--------------+                                                                                                                                                                                              
| uperf-client-10.179.0.7-eca3c4eb-mr956 | Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz |       x86_64       |      8       |                                                                                                                                                                                              
|     uperf-server-0-eca3c4eb-z2lcz      | Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz  |       x86_64       |      8       |                                                                                                                                                                                              
+----------------------------------------+------------------------------------------------+--------------------+--------------+                                                                                                                                                                                              

=================================================================== UUID: eca3c4eb-46f4-5136-adc7-c1acedcffd2e ===================================================================                                                                                                                                           
+----------------------------------------+----------------+                                                                                                                                                                                                                                                                  
|                 where                  | value.MemTotal |                                                                                                                                                                                                                                                                  
+----------------------------------------+----------------+                                                                                                                                                                                                                                                                  
| uperf-client-10.179.0.7-eca3c4eb-mr956 |  32449968 kB   |                                                                                                                                                                                                                                                                  
|     uperf-server-0-eca3c4eb-z2lcz      |  32105908 kB   |                                                                                                                                                                                                                                                                  
+----------------------------------------+----------------+                                                                                                                                                                                                                                                                  

=================================================================== UUID: eca3c4eb-46f4-5136-adc7-c1acedcffd2e ===================================================================                                                                                                                                           
+-------+-----------------------------------+                                                                                                                                                                                                                                                                                
| where |       value.cluster_version       |                                                                                                                                                                                                                                                                                
+-------+-----------------------------------+                                                                                                                                                                                                                                                                                
|  NA   | 4.8.0-0.nightly-2021-02-18-175130 |                                                                                                                                                                                                                                                                                
+-------+-----------------------------------+                                                                                                                                                                                                                                                                                
+-----------+----------+--------------+-------------------+-------------+-----------+----------------------------+--------------------------------------+--------------------+                                                                                                                                               
| test_type | protocol | message_size | read_message_size | num_threads | num_pairs |            key             |                 uuid                 |       value        |                                                                                                                                               
+-----------+----------+--------------+-------------------+-------------+-----------+----------------------------+--------------------------------------+--------------------+                                                                                                                                               
|  stream   |   tcp    |      64      |        64         |      1      |     4     |       max(norm_byte)       | a4afc882-101d-576d-a981-78c422d7aa97 |     53109760.0     |                                                                                                                                               
|  stream   |   tcp    |      64      |        64         |      1      |     4     |       avg(norm_byte)       | a4afc882-101d-576d-a981-78c422d7aa97 | 48491000.88888889  |                                                                                                                                               
|  stream   |   tcp    |      64      |        64         |      1      |     4     | 50.0percentiles(norm_byte) | a4afc882-101d-576d-a981-78c422d7aa97 |     48854528.0     |                                                                                                                                               
|  stream   |   tcp    |     1024     |       1024        |      1      |     4     |       max(norm_byte)       | a4afc882-101d-576d-a981-78c422d7aa97 |    531054592.0     |                                                                                                                                               
|  stream   |   tcp    |     1024     |       1024        |      1      |     4     |       avg(norm_byte)       | a4afc882-101d-576d-a981-78c422d7aa97 | 311397876.6180556  |                                                                                                                                               
|  stream   |   tcp    |     1024     |       1024        |      1      |     4     | 50.0percentiles(norm_byte) | a4afc882-101d-576d-a981-78c422d7aa97 |    310181888.0     |                                                                                                                                                                                                              
|  stream   |   tcp    |    16384     |       16384       |      1      |     4     |       max(norm_byte)       | a4afc882-101d-576d-a981-78c422d7aa97 |    619970560.0     |                                                                                                                                                                                                              
|  stream   |   tcp    |    16384     |       16384       |      1      |     4     |       avg(norm_byte)       | a4afc882-101d-576d-a981-78c422d7aa97 | 312442879.88055557 |                                                                                                                                                                                                              
|  stream   |   tcp    |    16384     |       16384       |      1      |     4     | 50.0percentiles(norm_byte) | a4afc882-101d-576d-a981-78c422d7aa97 | 297904810.6666667  |                                                                                                                                                                                                              
|  stream   |   udp    |     1024     |       1024        |      1      |     4     |       max(norm_byte)       | a4afc882-101d-576d-a981-78c422d7aa97 |    104579072.0     |                                                                                                                                                                                                              
|  stream   |   udp    |     1024     |       1024        |      1      |     4     |       avg(norm_byte)       | a4afc882-101d-576d-a981-78c422d7aa97 | 92939254.03059806  |                                                                                                                                                                                                              
|  stream   |   udp    |     1024     |       1024        |      1      |     4     | 50.0percentiles(norm_byte) | a4afc882-101d-576d-a981-78c422d7aa97 | 92757742.93333334  |                                                                                                                                                                                                              
|  stream   |   udp    |    16384     |       16384       |      1      |     4     |       max(norm_byte)       | a4afc882-101d-576d-a981-78c422d7aa97 |    838860800.0     |                                                                                                                                                                                                              
|  stream   |   udp    |    16384     |       16384       |      1      |     4     |       avg(norm_byte)       | a4afc882-101d-576d-a981-78c422d7aa97 | 751968267.3936023  |                                                                                                                                                                                                              
|  stream   |   udp    |    16384     |       16384       |      1      |     4     | 50.0percentiles(norm_byte) | a4afc882-101d-576d-a981-78c422d7aa97 | 752329448.7272727  |                                                                                                                                                                                                              
|  stream   |   udp    |      64      |        64         |      1      |     4     |       max(norm_byte)       | a4afc882-101d-576d-a981-78c422d7aa97 |     6422528.0      |                                                                                                                                                                                                              
|  stream   |   udp    |      64      |        64         |      1      |     4     |       avg(norm_byte)       | a4afc882-101d-576d-a981-78c422d7aa97 | 5901114.785185185  |                                                                                                                                                                                                              
|  stream   |   udp    |      64      |        64         |      1      |     4     | 50.0percentiles(norm_byte) | a4afc882-101d-576d-a981-78c422d7aa97 | 5904725.333333333  |                                                                                                                                                                                                              
+-----------+----------+--------------+-------------------+-------------+-----------+----------------------------+--------------------------------------+--------------------+                                                                                                                                                                                                              
+-----------+----------+--------------+-------------------+-------------+-----------+----------------------------+--------------------------------------+--------------------+                                                                                                                                                                                                              
| test_type | protocol | message_size | read_message_size | num_threads | num_pairs |            key             |                 uuid                 |       value        |                                                                                                                                                                                                              
+-----------+----------+--------------+-------------------+-------------+-----------+----------------------------+--------------------------------------+--------------------+                                                                                                                                                                                                              
|    rr     |   udp    |      64      |        64         |      1      |     4     |       max(norm_ops)        | a4afc882-101d-576d-a981-78c422d7aa97 |       3164.0       |                                                                                                                                                                                                              
|    rr     |   udp    |      64      |        64         |      1      |     4     |       avg(norm_ops)        | a4afc882-101d-576d-a981-78c422d7aa97 | 2970.233333333333  |                                                                                                                                                                                                              
|    rr     |   udp    |      64      |        64         |      1      |     4     | 90.0percentiles(norm_ltcy) | a4afc882-101d-576d-a981-78c422d7aa97 | 348.4944559733073  |                                                                                                                                                                                                              
|    rr     |   udp    |      64      |        64         |      1      |     4     | 99.0percentiles(norm_ltcy) | a4afc882-101d-576d-a981-78c422d7aa97 | 418.54228515624993 |                                                                                                                                                                                                              
|    rr     |   udp    |      64      |        64         |      1      |     4     |       avg(norm_ltcy)       | a4afc882-101d-576d-a981-78c422d7aa97 | 338.6982260386149  | 
etc...
+-----------+----------+--------------+-------------+----------------------------+--------------------------------------+--------------------+-------------+
| test_type | protocol | message_size | num_threads |            key             |                 uuid                 |       value        |  tolerancy  |
+-----------+----------+--------------+-------------+----------------------------+--------------------------------------+--------------------+-------------+
|    rr     |   tcp    |      64      |      1      | 99.0percentiles(norm_ltcy) | ec7f0cfb-0812-57ab-8905-fd6ddaacf593 | 346.40508117675773 |  baseline   |
|    rr     |   tcp    |      64      |      1      | 99.0percentiles(norm_ltcy) | 975fa650-aeb2-5042-8517-fe277d7cb1f3 |  382.039562988281  |     ok      |
|    rr     |   tcp    |     1024     |      1      | 99.0percentiles(norm_ltcy) | ec7f0cfb-0812-57ab-8905-fd6ddaacf593 | 351.03181762695306 |  baseline   |
|    rr     |   tcp    |     1024     |      1      | 99.0percentiles(norm_ltcy) | 975fa650-aeb2-5042-8517-fe277d7cb1f3 | 403.37683166503865 |     ok      |
|    rr     |   tcp    |    16384     |      1      | 99.0percentiles(norm_ltcy) | ec7f0cfb-0812-57ab-8905-fd6ddaacf593 | 392.5337493896483  |  baseline   |
|    rr     |   tcp    |    16384     |      1      | 99.0percentiles(norm_ltcy) | 975fa650-aeb2-5042-8517-fe277d7cb1f3 | 469.6159927368153  | failed: 15% |
|    rr     |   udp    |      64      |      1      | 99.0percentiles(norm_ltcy) | ec7f0cfb-0812-57ab-8905-fd6ddaacf593 | 361.8567254638668  |  baseline   |
|    rr     |   udp    |      64      |      1      | 99.0percentiles(norm_ltcy) | 975fa650-aeb2-5042-8517-fe277d7cb1f3 | 433.91663391113224 | failed: 15% |
|    rr     |   udp    |     1024     |      1      | 99.0percentiles(norm_ltcy) | ec7f0cfb-0812-57ab-8905-fd6ddaacf593 | 331.14006347656243 |  baseline   |
|    rr     |   udp    |     1024     |      1      | 99.0percentiles(norm_ltcy) | 975fa650-aeb2-5042-8517-fe277d7cb1f3 | 391.8874816894531  | failed: 15% |
|    rr     |   udp    |    16384     |      1      | 99.0percentiles(norm_ltcy) | ec7f0cfb-0812-57ab-8905-fd6ddaacf593 | 392.8086782836913  |  baseline   |
|    rr     |   udp    |    16384     |      1      | 99.0percentiles(norm_ltcy) | 975fa650-aeb2-5042-8517-fe277d7cb1f3 | 445.84954132080077 |     ok      |
+-----------+----------+--------------+-------------+----------------------------+--------------------------------------+--------------------+-------------+

Signed-off-by: Raul Sevilla rsevilla@redhat.com

rsevilla87 commented 3 years ago

With the latest commit, benchmark-comparison prints out the deviation. We can also avoid printing "good" results. thoughts?

+-------------+--------+----------------------+-----------+--------------------------+--------------------------------------+----------+----------------+                                                                                                                       
|  test_type  | routes | conn_per_targetroute | keepalive |           key            |                 uuid                 |  value   |   tolerancy    |                                                                                                                       
+-------------+--------+----------------------+-----------+--------------------------+--------------------------------------+----------+----------------+                                                                                                                       
|    http     |  500   |          1           |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 191160.0 |    baseline    |                                                                                                                       
|    http     |  500   |          1           |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 199434.0 |       ok       |                                                                                                                       
|    http     |  500   |          1           |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 12892.5  |    baseline    |                                                                                                                       
|    http     |  500   |          1           |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 11559.0  |       ok       |                                                                                                                       
|    http     |  500   |          1           |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 170566.0 |    baseline    |                                                                                                                       
|    http     |  500   |          1           |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 172050.5 |       ok       |                                                                                                                       
|    http     |  500   |          20          |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 66229.0  |    baseline    |                                                                                                                       
|    http     |  500   |          20          |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 72479.0  |       ok       |                                                                                                                       
|    http     |  500   |          20          |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 12097.5  |    baseline    |                                                                                                                       
|    http     |  500   |          20          |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 11075.0  |       ok       |                                                                                                                       
|    http     |  500   |          20          |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 75213.0  |    baseline    |                                                                                                                       
|    http     |  500   |          20          |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 89248.0  |       ok       |                                                                                                                       
|    http     |  500   |          80          |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 18093.5  |    baseline    |                                                                                                                       
|    http     |  500   |          80          |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 24754.0  |       ok       |                                                                                                                       
|    http     |  500   |          80          |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 |  8273.0  |    baseline    |                                                                                                                       
|    http     |  500   |          80          |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 11540.5  |       ok       |                                                                                                                       
|    http     |  500   |          80          |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 27612.5  |    baseline    |                                                                                                                       
|    http     |  500   |          80          |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 34349.5  |       ok       |                                                                                                                       
|    edge     |  500   |          1           |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 121735.5 |    baseline    |                                                                                                                       
|    edge     |  500   |          1           |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 125437.5 |       ok       |                                                                                                                       
|    edge     |  500   |          1           |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 11529.0  |    baseline    |                                                                                                                       
|    edge     |  500   |          1           |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 10835.0  |       ok       |                                                                                                                       
|    edge     |  500   |          1           |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 105690.5 |    baseline    |                                                                                                                       
|    edge     |  500   |          1           |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 107158.0 |       ok       |                                                                                                                       
|    edge     |  500   |          20          |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 29036.5  |    baseline    |                                                                                                                       
|    edge     |  500   |          20          |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 30339.0  |       ok       |                                                                                                                       
|    edge     |  500   |          20          |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 11440.0  |    baseline    |                                                                                                                       
|    edge     |  500   |          20          |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 10488.0  |       ok       |                                                                                                                       
|    edge     |  500   |          20          |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 29077.5  |    baseline    |                                                                                                                       
|    edge     |  500   |          20          |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 30408.0  |       ok       |                                                                                                                       
|    edge     |  500   |          80          |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 16973.0  |    baseline    |                                                                                                                       
|    edge     |  500   |          80          |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 21484.5  |       ok       |                                                                                                                       
|    edge     |  500   |          80          |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 |  4795.0  |    baseline    |                                                                                                                       
|    edge     |  500   |          80          |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 |  4526.5  |       ok       |                                                                                                                       
|    edge     |  500   |          80          |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 19626.0  |    baseline    |                                                                                                                       
|    edge     |  500   |          80          |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 18351.5  |       ok       |                                                                                                                       
| passthrough |  500   |          1           |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 244111.0 |    baseline    |                                                                                                                       
| passthrough |  500   |          1           |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 239246.0 |       ok       |                                                                                                                       
| passthrough |  500   |          1           |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 10501.0  |    baseline    |                                                                                                                       
| passthrough |  500   |          1           |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 |  9396.0  |       ok       |                                                                                                                       
| passthrough |  500   |          1           |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 197430.0 |    baseline    |                                                                                                                       
| passthrough |  500   |          1           |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 204720.5 |       ok       |                                                                                                                       
| passthrough |  500   |          20          |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 259266.0 |    baseline    |                                                                                                                       
| passthrough |  500   |          20          |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 263866.5 |       ok       |                                                                                                                       
| passthrough |  500   |          20          |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 10845.0  |    baseline    |                                                                                                                       
| passthrough |  500   |          20          |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 12305.0  |       ok       |                                                                                                                                                                    
| passthrough |  500   |          20          |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 235841.5 |    baseline    |                                                                                                                                                                    
| passthrough |  500   |          20          |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 242871.5 |       ok       |                                                                                                                                                                    
| passthrough |  500   |          80          |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 181174.5 |    baseline    |                                                                                                                                                                    
| passthrough |  500   |          80          |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 210086.5 |       ok       |                                                                                                                                                                    
| passthrough |  500   |          80          |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 |  8289.5  |    baseline    |                                                                                                                                                                    
| passthrough |  500   |          80          |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 10662.5  |       ok       |                                                                                                                                                                    
| passthrough |  500   |          80          |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 165855.5 |    baseline    |                                                                                                                                                                    
| passthrough |  500   |          80          |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 185107.5 |       ok       |                                                                                                                                                                    
|  reencrypt  |  500   |          1           |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 111811.5 |    baseline    |                                                                                                                                                                    
|  reencrypt  |  500   |          1           |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 113231.0 |       ok       |                                                                                                                                                                    
|  reencrypt  |  500   |          1           |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 |  8792.0  |    baseline    |                                                                                                                                                                    
|  reencrypt  |  500   |          1           |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 |  9054.0  |       ok       |                                                                                                                                                                    
|  reencrypt  |  500   |          1           |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 89626.0  |    baseline    |                                                                                                                                                                    
|  reencrypt  |  500   |          1           |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 93370.0  |       ok       |                                                                                                                                                                    
|  reencrypt  |  500   |          20          |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 11838.5  |    baseline    |                                                                                                                                                                    
|  reencrypt  |  500   |          20          |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 12305.5  |       ok       |                                                                                                                                                                    
|  reencrypt  |  500   |          20          |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 |  8235.5  |    baseline    |                                                                                                                                                                    
|  reencrypt  |  500   |          20          |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 |  8461.0  |       ok       |                                                                                                                                                                    
|  reencrypt  |  500   |          20          |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 11806.5  |    baseline    |                                                                                                                                                                    
|  reencrypt  |  500   |          20          |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 12062.5  |       ok       |                                                                                                                                                                    
|  reencrypt  |  500   |          80          |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 |  9483.5  |    baseline    |                                                                                                                                                                    
|  reencrypt  |  500   |          80          |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 |  9670.0  |       ok       |                                                                                                                                                                    
|  reencrypt  |  500   |          80          |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 |  3463.0  |    baseline    |                                                                                                                                                                    
|  reencrypt  |  500   |          80          |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 |  3142.5  |       ok       |                                                                                                                                                                    
|  reencrypt  |  500   |          80          |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 |  9154.5  |    baseline    |                                                                                                                                                                    
|  reencrypt  |  500   |          80          |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 |  8128.0  |       ok       |                                                                                                                                                                    
|     mix     |  2000  |          1           |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 152427.5 |    baseline    |                                                                                                                                                                    
|     mix     |  2000  |          1           |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 162476.5 |       ok       |                                                                                                                                                                                                                                   
|     mix     |  2000  |          1           |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 |  9789.0  |    baseline    |                                                                                                                                                                                                                                   
|     mix     |  2000  |          1           |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 11747.5  |       ok       |                                                                                                                                                                                                                                   
|     mix     |  2000  |          1           |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 136342.5 |    baseline    |                                                                                                                                                                                                                                   
|     mix     |  2000  |          1           |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 144521.5 |       ok       |                                                                                                                                                                                                                                   
|     mix     |  2000  |          10          |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 35576.0  |    baseline    |                                                                                                                                                                                                                                   
|     mix     |  2000  |          10          |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 41659.5  |       ok       |                                                                                                                                                                                                                                   
|     mix     |  2000  |          10          |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 11141.0  |    baseline    |                                                                                                                                                                                                                                   
|     mix     |  2000  |          10          |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 |  9238.0  | failed: 17.08% |                                                                                                                                                                                                                                   
|     mix     |  2000  |          10          |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 36587.5  |    baseline    |                                                                                                                                                                                                                                   
|     mix     |  2000  |          10          |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 36301.5  |       ok       |                                                                                                                                                                                                                                   
|     mix     |  2000  |          20          |     0     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 23357.5  |    baseline    |                                                                                                                                                                                                                                   
|     mix     |  2000  |          20          |     0     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 32097.0  |       ok       |                                                                                                                                                                                                                                   
|     mix     |  2000  |          20          |     1     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 10375.0  |    baseline    |                                                                                                                                                                                                                                   
|     mix     |  2000  |          20          |     1     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 |  8173.0  | failed: 21.22% |                                                                                                                                                                                                                                   
|     mix     |  2000  |          20          |    50     | avg(requests_per_second) | 6503a15d-ca27-4483-90d9-7116840aef87 | 22362.5  |    baseline    |                                                                                                                                                                                                                                   
|     mix     |  2000  |          20          |    50     | avg(requests_per_second) | fb9d3fb0-1050-48b7-8e6d-8e7d6b0ba319 | 25718.0  |       ok       |                                                                                                                                                                                                                                   
+-------------+--------+----------------------+-----------+--------------------------+--------------------------------------+----------+----------------+
jtaleric commented 3 years ago

Tested Locally, good to go!