[Mobile] Invalid graph with Node Resize (OpType:Resize) using QNN execution provider (DSP backend)

[aurelienpre]

Describe the issue

Hi, I am getting the following exception when creating an Ort::Session with the QNN Provider (DSP backend), and an ONNX model that makes use of Resize:

Node 'Resize' OpType:Resize with domain:com.ms.internal.nhwc was inserted using the NHWC format as requested by QNNExecutionProvider, but was not selected by that EP. This means the graph is now invalid as there will not be an EP able to run the node. This could be a bug in layout transformer, or in the GetCapability implementation of the EP.

This runs in the native (C++) part of an Android app, on a Samsung Tab S8 (Android 13) with a Snapdragon 8 Gen 1 (Hexagon architecture v69). QNN is supported for backend DSP on this device according to the qnn-platform-validator tool from the QNN SDK. The error was also reproduced on various other Android devices with different Snapdragon chips and Hexagon architectures. Other models not using the Resize operator work fine.

ONNX Runtime 1.15.0 was cross built on Linux for Android using QNN SDK v2.10.0.

ONNX Model to reproduce: dummy_net.zip

To reproduce

Python Create a ONNX model with the resize operator, for instance

class DummyNet(torch.nn.Module):
    def __init__(self):
        super(DummyNet, self).__init__()
        self.quant = QuantStub()
        self.dequant = DeQuantStub()

        self.name = 'dummey_net'
        self.input_names = ['input_image']
        self.output_names = ['class_logits']

        self.conv_1 = torch.nn.Conv2d(3, 32, 3, 3)
        self.conv_2 = torch.nn.Conv2d(32, 10, 3, 3)
        self.resize = torch.nn.Upsample(scale_factor=2., mode='nearest', align_corners=None)

    def forward(self, x):
        x = self.quant(x)
        x = self.conv_1(x)
        x = self.resize(x)
        x = self.conv_2(x)
        x = self.dequant(x)
        return x

See full script to generate the model: dummy_net_onnx.py.txt

C++

const Ort::Env ortEnv(ORT_LOGGING_LEVEL_VERBOSE, "my_app");

std::unordered_map<std::string, std::string> qnnOptions;
qnnOptions["backend_path"] = "libQnnHtp.so";

Ort::SessionOptions sessionOptions;
sessionOptions.SetIntraOpNumThreads(1);
sessionOptions.AppendExecutionProvider("QNN", qnnOptions);

try {
    Ort::Session ortSession(ortEnv, "model path", sessionOptions);
} catch (const Ort::Exception& e) {
    return std::string("Exception: ") + e.what();
}

ONNX Model to reproduce: dummy_net.zip

Urgency

Medium urgency, currently blocking QNN DSP backend execution with ONNX models using the Resize operator

Platform

Android

OS Version

13

ONNX Runtime Installation

Built from Source

Compiler Version (if 'Built from Source')

clang 9

Package Name (if 'Released Package')

None

ONNX Runtime Version or Commit ID

1.15.0

ONNX Runtime API

C++/C

Architecture

ARM64

Execution Provider

Other / Unknown

Execution Provider Library Version

QNN SDK v2.10.0

[dtrealm]

Same bug on snapdrgon 865

[kristoftunner]

@aurelienpre any solution? I am facing the same issue.

[aurelienpre]

Yeah, I worked around the issue by using bilinear resize mode rather then nearest, works on my end

[kristoftunner]

@aurelienpre thanks! It works for me as well

[skottmckay]

@HectorSVC Should the QNN EP be checking for the mode? Possibly it doesn't support nearest.

[aurelienpre]

I tried that again with ONNX Runtime 1.16.0 built with QNN 2.13.4 and I didn't run into the same error, closing the issue.

[Jamil]

I'm running into this now with both onnxruntime 1.18 and ort-nightly, when quantizing and running a YoloV8 model converted into ONNX.