PaddleOCR and NPU

[aropb]

Hi,

I have an Intel Core Ultra 9 Processor with NPU. It was not possible to start recognition for the NPU. Everything works for the CPU.

This my code:

        string ocrPath = publishDirectory + "/wwwroot/ocr/";
        var options = new Sdcb.OpenVINO.DeviceOptions("NPU", new Dictionary<string, string>
        {
            { Sdcb.OpenVINO.PropertyKeys.HintPerformanceMode, "THROUGHPUT" },
            { Sdcb.OpenVINO.PropertyKeys.HintInferencePrecision, "f16" },
            { Sdcb.OpenVINO.PropertyKeys.HintEnableCpuPinning, "NO" },
            { Sdcb.OpenVINO.PropertyKeys.NumStreams, "1" },
            { Sdcb.OpenVINO.PropertyKeys.EnableProfiling, "NO" }
        });

        OcrEng = new PaddleOcrRecognizer(RecognizationModel.FromDirectory(ocrPath + "english", ocrPath + "en_dict.txt", ModelVersion.V4), options, 192); // or dynamic shapes

I get an error when executing:

Exception "Sdcb.OpenVINO.OpenVINOException" in Sdcb.OpenVINO.dll: 'general error(-1) Exception from src\inference\src\cpp\core.cpp:109: Exception from src\inference\src\dev\plugin.cpp:54: Exception from src\plugins\intel_npu\src\plugin\src\plugin.cpp:513: get_shape was called on a descriptor::Tensor with dynamic shape

OpenVINO NPU documentations: https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/npu-device.html

Limitations Currently, only models with static shapes are supported on NPU.

Please tell me what to do? Thanks.

[sdcb]

You can specify static shapes by this code: https://github.com/sdcb/OpenVINO.NET/blob/35344c0b4adf901c7471e616236b106ea7fe8ca8/projects/PaddleOCR/Sdcb.OpenVINO.PaddleOCR/PaddleOcrOptions.cs#L27-L28

[aropb]

        string ocrPath = publishDirectory + "/wwwroot/ocr/";

        Sdcb.OpenVINO.DeviceOptions deviceOptions = new("NPU", new Dictionary<string, string>
        {
            { Sdcb.OpenVINO.PropertyKeys.HintPerformanceMode, "THROUGHPUT" },
            { Sdcb.OpenVINO.PropertyKeys.HintInferencePrecision, "f16" }, // i8 f16
            { Sdcb.OpenVINO.PropertyKeys.NumStreams, "1" },
            { Sdcb.OpenVINO.PropertyKeys.EnableProfiling, "NO" }
        });

        PaddleOcrOptions ocrOptions = new(deviceOptions)
        {
            DetectionStaticSize = new Size(480, 64),
            RecognitionStaticWidth = 480
        };

        OcrEng = new PaddleOcrAll(
            new FullOcrModel(
                DetectionModel.FromDirectory(ocrPath + "english", ModelVersion.V4), 
                RecognizationModel.FromDirectory(ocrPath + "english", ocrPath + "en_dict.txt", ModelVersion.V4)
            ),
            ocrOptions);

Error: Exception from src\inference\src\dev\plugin.cpp:54: Exception from src\plugins\intel_npu\src\plugin\src\plugin.cpp:513: get_shape was called on a descriptor::Tensor with dynamic shape

What am I doing wrong?

Besides, I would like to use only recognition: OcrEng = new PaddleOcrRecognizer(RecognizationModel.FromDirectory(ocrPath + "english", ocrPath + "en_dict.txt", ModelVersion.V4), options);

Thanks.