ROCm / AMDMIGraphX

AMD's graph optimization engine.
https://rocm.docs.amd.com/projects/AMDMIGraphX/en/latest/
MIT License
185 stars 86 forks source link

MI300 specific FP8 ASM fix #3547

Closed CharlieL7 closed 3 weeks ago

CharlieL7 commented 3 weeks ago

Disables the fp8 ASM on MI300 for non-FNUZ types. MI300 only has hardware support for FNUZ FP8 types. Fixes test_verify bugs seen on MI300.

migraphx-bot commented 3 weeks ago
Test Batch Rate new
25840d
Rate old
b73def
Diff Compare
torchvision-resnet50 64 3,260.10 3,257.93 0.07% :white_check_mark:
torchvision-resnet50_fp16 64 6,991.63 6,992.99 -0.02% :white_check_mark:
torchvision-densenet121 32 2,437.02 2,432.26 0.20% :white_check_mark:
torchvision-densenet121_fp16 32 4,105.40 4,038.39 1.66% :white_check_mark:
torchvision-inceptionv3 32 1,639.82 1,638.89 0.06% :white_check_mark:
torchvision-inceptionv3_fp16 32 2,764.70 2,761.69 0.11% :white_check_mark:
cadene-inceptionv4 16 776.41 776.39 0.00% :white_check_mark:
cadene-resnext64x4 16 811.61 811.37 0.03% :white_check_mark:
slim-mobilenet 64 7,536.52 7,532.73 0.05% :white_check_mark:
slim-nasnetalarge 64 211.56 211.42 0.06% :white_check_mark:
slim-resnet50v2 64 3,503.55 3,507.25 -0.11% :white_check_mark:
bert-mrpc-onnx 8 1,144.94 1,147.76 -0.25% :white_check_mark:
bert-mrpc-tf 1 464.28 469.91 -1.20% :white_check_mark:
pytorch-examples-wlang-gru 1 422.70 514.96 -17.92% :red_circle:
pytorch-examples-wlang-lstm 1 382.88 386.61 -0.97% :white_check_mark:
torchvision-resnet50_1 1 779.78 772.05 1.00% :white_check_mark:
cadene-dpn92_1 1 398.26 398.73 -0.12% :white_check_mark:
cadene-resnext101_1 1 383.39 383.67 -0.07% :white_check_mark:
onnx-taau-downsample 1 343.43 342.33 0.32% :white_check_mark:
dlrm-criteoterabyte 1 33.33 33.33 0.01% :white_check_mark:
dlrm-criteoterabyte_fp16 1 52.73 52.70 0.04% :white_check_mark:
agentmodel 1 8,371.77 8,056.20 3.92% :high_brightness:
unet_fp16 2 58.95 58.92 0.05% :white_check_mark:
resnet50v1_fp16 1 1,022.62 950.32 7.61% :high_brightness:
resnet50v1_int8 1 1,008.30 1,000.02 0.83% :white_check_mark:
bert_base_cased_fp16 64 1,169.99 1,169.24 0.06% :white_check_mark:
bert_large_uncased_fp16 32 363.81 363.69 0.03% :white_check_mark:
bert_large_fp16 1 199.75 198.89 0.43% :white_check_mark:
distilgpt2_fp16 16 2,203.59 2,203.09 0.02% :white_check_mark:
yolov5s 1 534.76 540.85 -1.13% :white_check_mark:
tinyllama 1 43.53 43.43 0.22% :white_check_mark:
vicuna-fastchat 1 169.82 170.64 -0.48% :white_check_mark:
whisper-tiny-encoder 1 418.02 418.21 -0.05% :white_check_mark:
whisper-tiny-decoder 1 430.16 426.10 0.95% :white_check_mark:

This build is not recommended to merge :red_circle:

migraphx-bot commented 3 weeks ago


     :white_check_mark: bert-mrpc-onnx: PASSED: MIGraphX meets tolerance
     :white_check_mark: bert-mrpc-tf: PASSED: MIGraphX meets tolerance
     :white_check_mark: pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance
     :white_check_mark: pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance
     :white_check_mark: torchvision-resnet50_1: PASSED: MIGraphX meets tolerance
     :white_check_mark: cadene-dpn92_1: PASSED: MIGraphX meets tolerance
     :white_check_mark: cadene-resnext101_1: PASSED: MIGraphX meets tolerance
     :white_check_mark: dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance
     :white_check_mark: agentmodel: PASSED: MIGraphX meets tolerance
     :white_check_mark: unet: PASSED: MIGraphX meets tolerance
     :white_check_mark: resnet50v1: PASSED: MIGraphX meets tolerance
     :white_check_mark: bert_base_cased_fp16: PASSED: MIGraphX meets tolerance
:red_circle:bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

     :white_check_mark: bert_large: PASSED: MIGraphX meets tolerance
     :white_check_mark: yolov5s: PASSED: MIGraphX meets tolerance
     :white_check_mark: tinyllama: PASSED: MIGraphX meets tolerance
     :white_check_mark: vicuna-fastchat: PASSED: MIGraphX meets tolerance
     :white_check_mark: whisper-tiny-encoder: PASSED: MIGraphX meets tolerance
     :white_check_mark: whisper-tiny-decoder: PASSED: MIGraphX meets tolerance
     :white_check_mark: distilgpt2_fp16: PASSED: MIGraphX meets tolerance

codecov[bot] commented 3 weeks ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 92.16%. Comparing base (3c3d007) to head (25840d4). Report is 1 commits behind head on develop.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #3547 +/- ## ======================================== Coverage 92.16% 92.16% ======================================== Files 512 512 Lines 21401 21401 ======================================== Hits 19724 19724 Misses 1677 1677 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.