search

openvinotoolkit / openvino

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
https://docs.openvino.ai
Apache License 2.0
7.06k stars 2.22k forks source link

Unable to inference IR on NCS2 because of ScatterNDupdate layers #11941

Closed ashishpapanai closed 2 years ago

ashishpapanai commented 2 years ago
System information (version)
Detailed description
Steps to reproduce

I've converted PyTorch Model to ONNX using PyTorch Exporter. The model has ScatterNDupdate layers, which isn't supported on NCS2. Is there any way to remove those layers or any alternative to it so that the model can be inference upon using NCS2?

Issue submission checklist
Iffa-Intel commented 2 years ago

@ashishpapanai did you get any error when inferencing the model with NCS2? If yes could you share it? Please help to share your ONNX model too, if possible.

ashishpapanai commented 2 years ago

Hi @Iffa-Meah, The error message is: [ ERROR ] Failed to compile layer "1068": unsupported layer type "ScatterNDUpdate" The ONNX model can be found here: link

Iffa-Intel commented 2 years ago

@ashishpapanai may I know which code I need to run to generate your .onnx file? because I can see varieties of custom codes in that link

ashishpapanai commented 2 years ago

@Iffa-Meah https://github.com/ashishpapanai/VisionLAN-OpenVino/blob/openvino/convertONNX.py This file handles the model conversion, and the model can be found here link

And based on my analysis (which might be incorrect), the splices resulting in the ScatterNDupdate layer is Line No: 94-112 of resnet.py

Iffa-Intel commented 2 years ago

@ashishpapanai , looking at your implementations in detail, I cross-checked them with OpenVINO official documentation to see compatibility/support. The ScatterNDUpdate layer is indeed unsupported for NCS2 since it is not listed in this Supported Layer here.

Your workaround is to create a custom layer for VPU that could replace the ScatterNDUpdate functionality. You may refer to this guide.

ashishpapanai commented 2 years ago

Hey @Iffa-Meah, Is there any other (simpler) workaround, rather than creating a Custom Kernel?

Iffa-Intel commented 2 years ago

@ashishpapanai I'm afraid not because it involves a layer that is not supported. It needs to be custom-made to make it compatible.

ashishpapanai commented 2 years ago

@Iffa-Meah, Is there any reason for creation of this layer (fucntionality it tries to implement), so that I can refractor the layer creating code to prevent creations of sctatterNDupdate and create any other supported layer (with similar function, with maybe reduced or unaffected efficiency) instead.

Iffa-Intel commented 2 years ago

@ashishpapanai this layer is created for data movement operations. The operation produces a copy of data tensor and updates its value to values specified by updates at specific index positions specified by indices.

To understand in detail, you need to refer here.

If you are able to find any supported layer that has similar functionality, then you are welcomed to try it.

Iffa-Intel commented 2 years ago

Closing issue, feel free to re-open or start a new issue if additional assistance is needed.

IkrameBeggar commented 2 months ago
System information (version)
  • OpenVINO => 2021.1
  • Operating System / Platform => Ubuntu 20.04 LTS
  • Compiler => Visual Studio 2017
  • Problem classification: Model Conversion
  • Framework: PyTorch
  • Model name: VisionLAN
Detailed description
Steps to reproduce

I've converted PyTorch Model to ONNX using PyTorch Exporter. The model has ScatterNDupdate layers, which isn't supported on NCS2. Is there any way to remove those layers or any alternative to it so that the model can be inference upon using NCS2?

Issue submission checklist
  • [x] I report the issue, it's not a question
  • [x] I checked the problem with documentation, FAQ, open issues, Stack Overflow, etc and have not found solution
  • [x] There is reproducer code and related data files: images, videos, models, etc.

Hi @ashishpapanai I am currently facing the same issue with ScatterNDUpdate. Did you arrive to solve it? Thank you