hzeller / flaschen-taschen

Noisebridge Flaschen Taschen display
GNU General Public License v3.0
195 stars 49 forks source link

Add Pytest benchmarks. #69

Closed HexDecimal closed 1 month ago

HexDecimal commented 2 years ago

I mentioned profiling earlier and decided to write some benchmarks. These use pytest-benchmark.

============================= test session starts =============================
platform win32 -- Python 3.10.5, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: C:\Users\4b796\Projects\_others_projects\flaschen-taschen\api\python
plugins: benchmark-3.4.1, cov-2.12.1
collected 7 items

test_flaschen.py .......                                                 [100%]

------------------------------------------------------------------------------------------------------- benchmark: 7 tests -------------------------------------------------------------------------------------------------------
Name (time in ns)                        Min                     Max                    Mean                 StdDev                  Median                    IQR            Outliers  OPS (Kops/s)            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_set                            439.9917 (1.0)        2,645.0027 (1.0)          461.4312 (1.0)          28.1379 (1.0)          459.9919 (1.0)          10.0001 (1.0)     3306;4980    2,167.1704 (1.0)       87721          20
test_send                         3,699.9118 (8.41)      27,799.9789 (10.51)      4,184.8972 (9.07)        765.3272 (27.20)      4,000.0305 (8.70)        299.6530 (29.97)     598;686      238.9545 (0.11)      10289           1
test_send_array_transparent       5,799.8113 (13.18)    146,599.9521 (55.43)      6,799.1550 (14.73)     3,027.1605 (107.58)     6,299.9316 (13.70)       299.8859 (29.99)    189;2150      147.0771 (0.07)      13948           1
test_send_array                  47,299.7781 (107.50)   199,300.0042 (75.35)     50,716.3139 (109.91)    5,579.8516 (198.30)    49,399.9105 (107.39)    1,899.8981 (189.99)    409;469       19.7175 (0.01)       4622           1
test_send_new_array              50,200.1494 (114.09)   193,000.0726 (72.97)     53,330.9120 (115.58)    3,547.6356 (126.08)    52,800.1692 (114.79)    1,299.8935 (129.99)    334;451       18.7509 (0.01)       6072           1
test_flaschen_init              185,099.8960 (420.69)   433,399.8077 (163.86)   205,587.9331 (445.54)   24,393.0880 (866.91)   199,100.0026 (432.83)   10,499.9635 (>1000.0)     33;41        4.8641 (0.00)        323           1
test_set_full                   657,600.1178 (>1000.0)  885,799.9928 (334.90)   682,893.4062 (>1000.0)  18,145.3584 (644.87)   681,199.9483 (>1000.0)  17,100.0138 (>1000.0)    150;21        1.4644 (0.00)       1334           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
============================== 7 passed in 3.96s ==============================

It seems transparency handing is a big factor in the performance of send_array. Ignoring that it's about twice as slow as sending _data.