search

rapidsai / crossfit

Metric calculation library
Apache License 2.0
2 stars 6 forks source link

remove beir dependency #30

Closed edknv closed 11 months ago

github-actions[bot] commented 11 months ago

Benchmark comparison for 5f0a189 (main) vs 3a4e8f62 (PR)


--------------------------------------------------------------------------------------------- benchmark: 2 tests ---------------------------------------------------------------------------------------------
Name (time in s)                                  Min                 Max                Mean            StdDev             Median               IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_fiqa_all_MiniLM_L6_v2 (0002_3a4e8f6)     98.5032 (1.0)      103.2887 (1.01)     100.3752 (1.00)     2.0957 (2.01)     99.1543 (1.0)      3.3354 (2.94)          1;0  0.0100 (1.00)          5           1
test_fiqa_all_MiniLM_L6_v2 (0001_main)        99.3080 (1.01)     102.0415 (1.0)      100.3161 (1.0)      1.0435 (1.0)      99.9948 (1.01)     1.1345 (1.0)           1;0  0.0100 (1.0)           5           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean