search

rapidsai / crossfit

Metric calculation library
Apache License 2.0
2 stars 5 forks source link

upgrade base container to nemo:23.10 #42

Closed edknv closed 8 months ago

github-actions[bot] commented 8 months ago

Benchmark comparison for 3da5f6f (main) vs e8387b3c (PR)


---------------------------------------------------------------------------------------------- benchmark: 2 tests ----------------------------------------------------------------------------------------------
Name (time in s)                                   Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_fiqa_all_MiniLM_L6_v2 (0001_main)         97.2569 (1.0)      102.4196 (1.0)       99.4226 (1.0)      2.0761 (1.97)      99.3209 (1.0)      3.2152 (2.08)          2;0  0.0101 (1.0)           5           1
test_fiqa_all_MiniLM_L6_v2 (0002_e8387b3)     101.5733 (1.04)     104.1472 (1.02)     102.5928 (1.03)     1.0519 (1.0)      102.7328 (1.03)     1.5453 (1.0)           1;0  0.0097 (0.97)          5           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
github-actions[bot] commented 8 months ago

Benchmark comparison for 3da5f6f (main) vs 324a64df (PR)


---------------------------------------------------------------------------------------------- benchmark: 2 tests ---------------------------------------------------------------------------------------------
Name (time in s)                                  Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_fiqa_all_MiniLM_L6_v2 (0001_main)        97.1649 (1.0)      103.3715 (1.0)      100.3912 (1.0)      2.7153 (1.0)      100.1716 (1.00)     4.9129 (1.0)           2;0  0.0100 (1.0)           5           1
test_fiqa_all_MiniLM_L6_v2 (0002_324a64d)     97.7439 (1.01)     106.3620 (1.03)     101.7532 (1.01)     3.9562 (1.46)      99.7031 (1.0)      6.9021 (1.40)          2;0  0.0098 (0.99)          5           1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
github-actions[bot] commented 8 months ago

Benchmark comparison for 3da5f6f (main) vs 2f83d740 (PR)


---------------------------------------------------------------------------------------------- benchmark: 2 tests ----------------------------------------------------------------------------------------------
Name (time in s)                                   Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_fiqa_all_MiniLM_L6_v2 (0001_main)         98.5156 (1.0)      106.3430 (1.01)     102.2342 (1.0)      2.8741 (1.92)     102.3993 (1.0)      3.5119 (1.65)          2;0  0.0098 (1.0)           5           1
test_fiqa_all_MiniLM_L6_v2 (0002_2f83d74)     101.2285 (1.03)     105.1030 (1.0)      102.8694 (1.01)     1.4994 (1.0)      102.6874 (1.00)     2.1273 (1.0)           2;0  0.0097 (0.99)          5           1
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean