MatMulInteger op can accept different input types for T1 and T2 inputs, but MIGraphX only accepts in8 input tensors.
MIGraphx handles MatMulInteger in two files: parse_matmul.cpp and in quant_dot.hpp. In the second file there is a restriction about that T1 and T2 should be the same type, but it's not the case.
I came across this when tried to compile BERT-Squad int8 onnx zoo model:
migraphx-driver compile text/machine_comprehension/bert-squad/model/bertsquad-12-int8.onnx with the changes from this PR
Got the following error:
what(): /code/AMDMIGraphX/src/include/migraphx/check_shapes.hpp:210: same_type: quant_dot: Types do not match
Also the a_zero_point and b_zero_point optional inputs are not handled in MIGraphX.
The list of actions required:
[x] Update MatMulInteger implementation to accept different types for T1 and T2 inputs
[x] Update MatMulInteger implementation to process a_zero_point and b_zero_point
gyulaz-htec
commented
6 months ago
MatMulInteger op can accept different input types for T1 and T2 inputs, but MIGraphX only accepts in8 input tensors. MIGraphx handles MatMulInteger in two files: parse_matmul.cpp and in quant_dot.hpp. In the second file there is a restriction about that T1 and T2 should be the same type, but it's not the case.
I came across this when tried to compile BERT-Squad int8 onnx zoo model:
migraphx-driver compile text/machine_comprehension/bert-squad/model/bertsquad-12-int8.onnxwith the changes from this PR Got the following error:what(): /code/AMDMIGraphX/src/include/migraphx/check_shapes.hpp:210: same_type: quant_dot: Types do not matchAlso thea_zero_pointandb_zero_pointoptional inputs are not handled in MIGraphX. The list of actions required:a_zero_pointandb_zero_point