search

ThanatosShinji / onnx-tool

A parser, editor and profiler tool for ONNX models.
https://pypi.org/project/onnx-tool/
MIT License
383 stars 51 forks source link

Support for MultiScaleDeformableAttn module #73

Open Thaslim opened 6 months ago

Thaslim commented 6 months ago

I am trying to profile BEVFormer which has MultiScaleDeformableAttn module, current tool throws error: NotImplementedError: this Node MultiScaleDeformableAttnTRT2-MultiScaleDeformableAttnTRT2_2825 has no value_infer Is there any workaround/ plan to support Deformable attention layer. I am mainly looking to get list of OPs and MAC/ Param counts.

ThanatosShinji commented 6 months ago

You should profile this model with custom-registered nodes, refer this

Thaslim commented 6 months ago

Hi I get the following error when trying the example mentioned using @NODE_REGISTRY When I try to import NODEPROFILER_REGISTRY, it throws an error

/home/thasleem.tajudeen/miniconda3/envs/cramnet/lib/python3.9/site-packages/onnx_tool/node.py:2511: UserWarning: node MultiScaleDeformableAttnTRT2 is not registed for profiling, return 0 Macs and 0 params as default. Use NODEPROFILER_REGISTRY to register your profiler for this node.
  warnings.warn(f'node {n.op_type} is not registed for profiling, return 0 Macs and 0 params as default. '
Node Init Time Elapsed 0.007455110549926758
Tensor Init Time Elapsed 0.11277580261230469
IO Tensor Init Time Elapsed 0.0007681846618652344
Traceback (most recent call last):
  File "/home/thasleem.tajudeen/enntools-workspace/BFCP/bfcp_head/bev.py", line 44, in <module>
    bevformer()
  File "/home/thasleem.tajudeen/enntools-workspace/BFCP/bfcp_head/bev.py", line 37, in bevformer
    m = onnx_tool.Model(file,{'verbose':True,"constant_folding":True})
  File "/home/thasleem.tajudeen/miniconda3/envs/cramnet/lib/python3.9/site-packages/onnx_tool/model.py", line 26, in __init__
    self.graph = Graph(m.graph, self.cfg)
  File "/home/thasleem.tajudeen/miniconda3/envs/cramnet/lib/python3.9/site-packages/onnx_tool/graph.py", line 184, in __init__
    self.__constant_search__(self.cfg.constant_folding)
  File "/home/thasleem.tajudeen/miniconda3/envs/cramnet/lib/python3.9/site-packages/onnx_tool/graph.py", line 290, in __constant_search__
    this_node.value_infer(itensors, otensors)
  File "/home/thasleem.tajudeen/miniconda3/envs/cramnet/lib/python3.9/site-packages/onnx_tool/node.py", line 158, in value_infer
    raise NotImplementedError(f'this Node {self.op_type}-{self.name} has no value_infer')
NotImplementedError: this Node MultiScaleDeformableAttnTRT2-MultiScaleDeformableAttnTRT2_2825 has no value_infer
ThanatosShinji commented 6 months ago

@Thaslim It seems that you used a different ONNX model. In my example, the TRT operator is MultiScaleDeformableAttnTRT, but you got MultiScaleDeformableAttnTRT2.

Thaslim commented 3 months ago

Hi @ThanatosShinji ,The ONNX model I have uses this TRT plugin

ThanatosShinji commented 3 months ago

how to get this model?