Closed furiosamg closed 1 year ago
42ca703 ci: add github-add-comment tkn command
----------------------------------------------------- benchmark: 1 tests ----------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------
test_mlcommons_resnet50_accuracy 1.1273 82.6663 6.4307 6.0610 4.3001 1.6457 4745;7011 155.5042 50000 1
-----------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
42ca703 ci: add github-add-comment tkn command
-------------------------------------------------------------------------------------------------------- benchmark: 2 tests -------------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_mlcommons_ssd_resnet34_with_native_rust_pp_accuracy 40.6402 (1.0) 645.3738 (1.0) 61.8449 (1.0) 17.7657 (1.0) 61.2273 (1.0) 4.2201 (1.0) 40;248 16.1695 (1.0) 5000 1
test_mlcommons_ssd_resnet34_accuracy 93.5940 (2.30) 1,203.0367 (1.86) 144.1386 (2.33) 33.0366 (1.86) 143.4377 (2.34) 20.0884 (4.76) 522;250 6.9378 (0.43) 5000 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
738d39d fix: update accuracies
----------------------------------------------------------------------------------------------------- benchmark: 2 tests ----------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_mlcommons_ssd_mobilenet_with_native_rust_pp_accuracy 5.4056 (1.0) 23.3523 (1.0) 12.0214 (1.0) 3.0518 (1.0) 12.1962 (1.0) 4.8463 (1.27) 1899;1 83.1846 (1.0) 5000 1
test_mlcommons_ssd_mobilenet_accuracy 7.2928 (1.35) 277.6437 (11.89) 17.1936 (1.43) 4.7074 (1.54) 17.5479 (1.44) 3.8182 (1.0) 552;42 58.1611 (0.70) 5000 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
738d39d fix: update accuracies
----------------------------------------------------- benchmark: 1 tests ----------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------
test_efficientnetv2s_accuracy 6.2004 1,031.9957 13.2879 7.5174 12.7080 2.8159 818;1620 75.2566 50000 1
-----------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
738d39d fix: update accuracies
---------------------------------------------------- benchmark: 1 tests ---------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------
test_efficientnetb0_accuracy 4.2670 438.7789 13.3492 8.0267 12.4042 4.1395 1165;1187 74.9111 50000 1
---------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
738d39d fix: update accuracies
------------------------------------------------------ benchmark: 1 tests -----------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-------------------------------------------------------------------------------------------------------------------------------
test_mlcommons_resnet50_accuracy 1.0971 143.3437 9.7180 13.3905 4.6725 1.8744 5838;8380 102.9018 50000 1
-------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
6632ae3 fix: update yolov5l accuracy
----------------------------------------------------------------------------------------------------- benchmark: 2 tests ----------------------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_mlcommons_ssd_mobilenet_with_native_rust_pp_accuracy 4.6429 (1.0) 25.1739 (1.0) 10.0829 (1.0) 3.5788 (1.00) 8.1501 (1.0) 6.9012 (3.88) 1784;1 99.1782 (1.0) 5000 1
test_mlcommons_ssd_mobilenet_accuracy 8.4571 (1.82) 215.2643 (8.55) 18.3361 (1.82) 3.5628 (1.0) 18.5733 (2.28) 1.7785 (1.0) 453;463 54.5373 (0.55) 5000 1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
0.9.1 버전을 급하게 릴리즈 하느라 branch-0.9.1 에서만 작업하고 관련 내용이 main 에 머지되지 않았었습니다. 그 외에, 우리 sdk 말고 외부 파이썬 패키지들의 변경 때문에 정확도 변경이 있었는데요, ci를 성공하기 위해 반영했습니다.
리뷰 받을 준비 완료되어 레디로 바꾸겠습니다! ㅎㅎ