search

rapidsai / crossfit

Metric calculation library
Apache License 2.0
2 stars 6 forks source link

add an example beir report script #35

Closed edknv closed 11 months ago

github-actions[bot] commented 11 months ago

Benchmark comparison for 4e82341 (main) vs 04426bb8 (PR)


---------------------------------------------------------------------------------------------- benchmark: 2 tests ----------------------------------------------------------------------------------------------
Name (time in s)                                   Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_fiqa_all_MiniLM_L6_v2 (0001_main)         98.9540 (1.0)      101.3541 (1.0)       99.9158 (1.0)      0.8893 (1.24)      99.7036 (1.0)      0.9171 (1.0)           2;0  0.0100 (1.0)           5           1
test_fiqa_all_MiniLM_L6_v2 (0002_04426bb)     100.3842 (1.01)     102.0753 (1.01)     101.2436 (1.01)     0.7154 (1.0)      101.3327 (1.02)     1.2493 (1.36)          2;0  0.0099 (0.99)          5           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean