[Bug]: OpenVINO does not support the following ONNX operations: SparseConvolution, ScatterDense

shawn9977 commented 3 weeks ago

OpenVINO Version

2024.0/2024.1

Operating System

Ubuntu 20.04 (LTS)

Device used for inference

CPU

Framework

None

Model used

No response

Issue description

Can not convert static ONNX model to OV IR model with the command
ovc lidar.backbone.xyz.onnx --input [1,5]

Does this support ? or it is a bug ?

lidar.backbone.xyz.onnx is static model

Inputs: Name: 0, Shape: [1, 5]

Outputs: Name: 40, Shape: [1, 256, 180, 180]

root@14007861692c:/datav/CUDA-BEVFusion/model/resnet50# ovc lidar.backbone.xyz.onnx --input [1,5] [ ERROR ] ------------------------------------------------- [ ERROR ] ----------------- INTERNAL ERROR ---------------- [ ERROR ] Unexpected exception happened. [ ERROR ] Please verify parameters and environment. [ ERROR ] If you think this is a bug, please create new ticket here: [ ERROR ] https://github.com/openvinotoolkit/openvino/issues. [ ERROR ] -------------- DETAILED INFORMATION ------------- [ ERROR ] Check 'false' failed at src/frontends/onnx/frontend/src/frontend.cpp:144: FrontEnd API failed with GeneralFailure: OpenVINO does not support the following ONNX operations: SparseConvolution, ScatterDense Errors during ONNX translation: [ONNX Frontend] Conversion failed for Reshape--1 While validating ONNX node '<Node(Reshape): reshape0>': Node (reshape0): unknown attribute 'shape'

[ ERROR ] Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/openvino/tools/ovc/convert_impl.py", line 494, in _convert ov_model = driver(argv, {"conversion_parameters": non_default_params}) File "/usr/local/lib/python3.8/dist-packages/openvino/tools/ovc/convert_impl.py", line 250, in driver ov_model = moc_emit_ir(prepare_ir(argv), argv) File "/usr/local/lib/python3.8/dist-packages/openvino/tools/ovc/convert_impl.py", line 194, in prepare_ir ov_model = moc_pipeline(argv, moc_front_end) File "/usr/local/lib/python3.8/dist-packages/openvino/tools/ovc/moc_frontend/pipeline.py", line 296, in moc_pipeline ov_model = moc_front_end.convert(input_model) File "/usr/local/lib/python3.8/dist-packages/openvino/frontend/frontend.py", line 18, in convert converted_model = super().convert(model) openvino._pyopenvino.GeneralFailure: Check 'false' failed at src/frontends/onnx/frontend/src/frontend.cpp:144: FrontEnd API failed with GeneralFailure: OpenVINO does not support the following ONNX operations: SparseConvolution, ScatterDense Errors during ONNX translation: [ONNX Frontend] Conversion failed for Reshape--1 While validating ONNX node '<Node(Reshape): reshape0>': Node (reshape0): unknown attribute 'shape'

[ ERROR ] ----------------- END OF REPORT ----------------- [ ERROR ] -------------------------------------------------

Step-by-step reproduction

No response

Relevant log output

No response

Issue submission checklist

[X] I'm reporting an issue. It's not a question.
[X] I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
[X] There is reproducer code and related data files such as images, videos, models, etc.

andrei-kochin commented 3 weeks ago

Hello @shawn9977 thank you for reaching the OpenVINO!

This is not a bug as per error trace OpenVINO does not support the following ONNX operations: SparseConvolution, ScatterDense

Let us estimate the effort and @gkrivor we'll let you know would be a low hanging fruit or a long journey.

shawn9977 commented 2 weeks ago

Thanks for help @andrei-kochin @gkrivor could you please give a short answer ? is it a low hanging fruit or a long journey? Thank you!

gkrivor commented 2 weeks ago

Hi @shawn9977, could you provide which domain it is using? I don't see such operations in ONNX documentation https://github.com/onnx/onnx/blob/main/docs/Operators.md Anyway, I don't think we are planning support a custom domain. And I don't know about plans to support operations on a sparse tensors, yet. So, yes, you may expect looooong journey. Even now I can only imagine a conversion of a sparse tensor to dense tensor and applying a regular convolution. As you may expect - it may produce slightly unexpected performance results.

openvinotoolkit / openvino