search

rapidsai / crossfit

Metric calculation library
Apache License 2.0
2 stars 6 forks source link

handle 1d outputs from custom pytorch models #45

Closed edknv closed 9 months ago

github-actions[bot] commented 9 months ago

Benchmark comparison for 382a6d8 (main) vs 6f03f597 (PR)


---------------------------------------------------------------------------------------------- benchmark: 2 tests ---------------------------------------------------------------------------------------------
Name (time in s)                                  Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_fiqa_all_MiniLM_L6_v2 (0001_main)        99.2222 (1.0)      105.8137 (1.01)     101.8252 (1.0)      2.6432 (1.23)     101.7152 (1.0)      3.8823 (1.42)          1;0  0.0098 (1.0)           5           1
test_fiqa_all_MiniLM_L6_v2 (0002_6f03f59)     99.2945 (1.00)     104.7682 (1.0)      102.7605 (1.01)     2.1466 (1.0)      103.3234 (1.02)     2.7391 (1.0)           1;0  0.0097 (0.99)          5           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean