search

ThanatosShinji / onnx-tool

A parser, editor and profiler tool for ONNX models.
https://pypi.org/project/onnx-tool/
MIT License
383 stars 51 forks source link

Add Op MultiHeadAttention and GroupNorm Support #61

Open elonw2008 opened 9 months ago

elonw2008 commented 9 months ago

we find some error in some special model:

File "C:\Users\??\anaconda3\Lib\site-packages\onnx_tool\node.py", line 158, in value_infer raise NotImplementedError(f'this Node {self.op_type}-{self.name} has no value_infer') NotImplementedError: this Node MultiHeadAttention-MultiHeadAttention_0 has no value_infer

raise NotImplementedError(f'this Node {self.op_type}-{self.name} has no value_infer')

NotImplementedError: this Node GroupNorm-GroupNorm_0 has no value_infer

ThanatosShinji commented 9 months ago

Hi, I can not find these two operators in ONNX operators. Are they the custom operators created by you?

elonw2008 commented 9 months ago

Hi . not my custom operators . I convert the Pytorch unwayml/stable-diffusion-v1-5 model to onnxruntime_directml mode by Microsoft Olive . I find there are two operators don't support value infer and profiling Here is the link: https://onnx.ai/onnx/operators/onnx__GroupNormalization.html https://github.com/microsoft/onnxruntime/blob/1c6cb5dfebe38b09627430f0add54416653a2ceb/onnxruntime/contrib_ops/cpu/bert/multihead_attention.h#L13

ThanatosShinji commented 9 months ago

@elonw2008 GroupNormalization (GroupNorm you wrote, I believe, is ONNXRuntime's GroupNorm) is an ONNX operator. But multihead_attention is an operator of ONNXRuntime. It's more like a custom operator of the runtime library. Other runtime libraries may not have this operator.

ThanatosShinji commented 9 months ago

So I will suggest you add these OPs with onnx-tool's custom registry: code. I hope you understand that onnx-tool is a tool of ONNX, not ONNXRuntime.