Examples of notebooks/pixart cannot convert pixart model due to `to_shape was called on a dynamic shape.`

[peterzheng98]

Describe the bug When compiling to NPU, runtime error raised.

--> compiled_model = ov.compile_model(converted_model, device_name='NPU')

RuntimeError: Exception from src/inference/src/cpp/core.cpp:104:
Exception from src/inference/src/dev/plugin.cpp:53:
Exception from src/plugins/intel_npu/src/plugin/src/plugin.cpp:674:
Exception from src/plugins/intel_npu/src/plugin/src/compiled_model.cpp:61:
Exception from src/core/src/partial_shape.cpp:266:
to_shape was called on a dynamic shape.

The model remains unchanged (PixArt-alpha/PixArt-LCM-XL-2-1024-MS).

Installation instructions (Please mark the checkbox) [ x ] I followed the installation guide at https://github.com/openvinotoolkit/openvino_notebooks#-installation-guide to install the notebooks.

Environment information OpenVINO IE version: 2024.3.0-36-5e5f3726bef Python version: 3.10 torch version: 2.4.0 transformers version: 4.44.2

[brmarkus]

Have a look under "https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/npu-device.html#limitations", the NPU plugin currently only supports static shapes.

Can you run your use case on any other device type (CPU, GPU, AUTO, MULTI)?

[eaidova]

@peterzheng98
unfortunatly, NPU does not support dynamic shapes yet as stated in your error message. You can try to reshape model to static input shapes like demonstrated in this notebook https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/openvino-api/openvino-api.ipynb

@dmatveev @smirnov-alexey do you have info about pixart model support on NPU?

[smirnov-alexey]

@peterzheng98 unfortunatly, NPU does not support dynamic shapes yet as stated in your error message. You can try to reshape model to static input shapes like demonstrated in this notebook https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/openvino-api/openvino-api.ipynb

@dmatveev @smirnov-alexey do you have info about pixart model support on NPU?

I don't think so. I could run it on NPU once the shape is static to see if it helps

[peterzheng98]

Have a look under "https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/npu-device.html#limitations", the NPU plugin currently only supports static shapes.

Can you run your use case on any other device type (CPU, GPU, AUTO, MULTI)?

Thanks for the reply. The auto and GPU works.

[peterzheng98]

@peterzheng98 unfortunatly, NPU does not support dynamic shapes yet as stated in your error message. You can try to reshape model to static input shapes like demonstrated in this notebook https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/openvino-api/openvino-api.ipynb @dmatveev @smirnov-alexey do you have info about pixart model support on NPU?

I don't think so. I could run it on NPU once the shape is static to see if it helps

I've tried compiling the model into IR and visualizing the IR by netron. It seems that there's no -1 or ? in the parameters shape, and only the input shape contains ? or -1. Does it mean that I need change all -1 or ? to fixed value to inference that model on NPU?