Closed pfultz2 closed 2 weeks ago
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 92.16%. Comparing base (
c3a5367
) to head (76812a5
). Report is 9 commits behind head on develop.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Test | Batch | Rate new 76812a |
Rate old 71fd27 |
Diff | Compare | |
---|---|---|---|---|---|---|
torchvision-resnet50 | 64 | 3,256.57 | 3,257.89 | -0.04% | :white_check_mark: | |
torchvision-resnet50_fp16 | 64 | 6,992.90 | 6,970.17 | 0.33% | :white_check_mark: | |
torchvision-densenet121 | 32 | 2,433.77 | 2,434.31 | -0.02% | :white_check_mark: | |
torchvision-densenet121_fp16 | 32 | 4,087.53 | 4,075.92 | 0.28% | :white_check_mark: | |
torchvision-inceptionv3 | 32 | 1,636.70 | 1,638.52 | -0.11% | :white_check_mark: | |
torchvision-inceptionv3_fp16 | 32 | 2,764.24 | 2,764.44 | -0.01% | :white_check_mark: | |
cadene-inceptionv4 | 16 | 777.00 | 775.75 | 0.16% | :white_check_mark: | |
cadene-resnext64x4 | 16 | 811.62 | 808.04 | 0.44% | :white_check_mark: | |
slim-mobilenet | 64 | 7,535.36 | 7,535.82 | -0.01% | :white_check_mark: | |
slim-nasnetalarge | 64 | 211.42 | 211.48 | -0.03% | :white_check_mark: | |
slim-resnet50v2 | 64 | 3,504.08 | 3,506.62 | -0.07% | :white_check_mark: | |
bert-mrpc-onnx | 8 | 1,150.10 | 1,150.10 | 0.00% | :white_check_mark: | |
bert-mrpc-tf | 1 | 463.87 | 473.40 | -2.01% | :white_check_mark: | |
pytorch-examples-wlang-gru | 1 | 415.42 | 416.64 | -0.29% | :white_check_mark: | |
pytorch-examples-wlang-lstm | 1 | 399.13 | 401.17 | -0.51% | :white_check_mark: | |
torchvision-resnet50_1 | 1 | 777.41 | 751.90 | 3.39% | :high_brightness: | |
cadene-dpn92_1 | 1 | 399.55 | 397.07 | 0.63% | :white_check_mark: | |
cadene-resnext101_1 | 1 | 374.03 | 383.71 | -2.52% | :white_check_mark: | |
onnx-taau-downsample | 1 | 343.48 | 343.32 | 0.05% | :white_check_mark: | |
dlrm-criteoterabyte | 1 | 33.35 | 33.32 | 0.11% | :white_check_mark: | |
dlrm-criteoterabyte_fp16 | 1 | 52.69 | 52.76 | -0.13% | :white_check_mark: | |
agentmodel | 1 | 8,300.13 | 8,370.16 | -0.84% | :white_check_mark: | |
unet_fp16 | 2 | 59.00 | 58.90 | 0.16% | :white_check_mark: | |
resnet50v1_fp16 | 1 | 939.06 | 941.27 | -0.23% | :white_check_mark: | |
resnet50v1_int8 | 1 | 1,003.48 | 1,005.65 | -0.22% | :white_check_mark: | |
bert_base_cased_fp16 | 64 | 1,171.94 | 1,171.62 | 0.03% | :white_check_mark: | |
bert_large_uncased_fp16 | 32 | 363.41 | 363.80 | -0.11% | :white_check_mark: | |
bert_large_fp16 | 1 | 198.61 | 198.85 | -0.12% | :white_check_mark: | |
distilgpt2_fp16 | 16 | 2,204.23 | 2,203.72 | 0.02% | :white_check_mark: | |
yolov5s | 1 | 531.50 | 533.95 | -0.46% | :white_check_mark: | |
tinyllama | 1 | 43.51 | 43.43 | 0.18% | :white_check_mark: | |
vicuna-fastchat | 1 | 177.40 | 176.68 | 0.40% | :white_check_mark: | |
whisper-tiny-encoder | 1 | 418.09 | 418.65 | -0.13% | :white_check_mark: | |
whisper-tiny-decoder | 1 | 428.40 | 437.27 | -2.03% | :white_check_mark: |
Check results before merge :high_brightness:
:red_circle:bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output
This will time the model in a loop without synchronization until after the loop. This should get much closer to actual device time, and its useful when doing timing on small subgraphs(since they usually run less then 100 microseconds).