[Mobile] QNN SetupBackend failed qnn_backend_manager.cc:288 InitializeBackend Failed to initialize backend

[bibekanandaburagohain]

The issue

I have compiled onnxruntime with qnn backend support for arm64-v8a android api 34. My final goal is to use this in sherpa onnx (k2) encoder decoder model to run in qnn htp backend. But while running the app, on SM8650P processor, I am getting the below error - [E:onnxruntime:, qnn_execution_provider.cc:499 GetCapability] QNN SetupBackend failed qnn_backend_manager.cc:288 InitializeBackend Failed to initialize backend

Can anyone help me with this. Thanks in Advance.

To reproduce

First built the onnxruntime with qnn support - ./build.sh --build_shared_lib --skip_submodule_sync --android --config Release --use_qnn --qnn_home <QNN_PATH> --android_sdk_path <SDK_PATH> --android_ndk_path <NDK_PATH> --android_abi arm64-v8a --android_api 34 --cmake_generator Ninja --build_dir build/Android

Then used the libonnxruntime.so and headers in building of sherpa-onnx android binaries. Also made following changes to uses qnn EP - std::unordered_map<std::string, std::string> qnn_options; qnn_options["backend_path"] = "libQnnHtp.so"; sess_opts_.AddConfigEntry(kOrtSessionOptionEpContextEnable, "1"); sess_opts_.AppendExecutionProvider("QNN", qnn_options);

Platform

Android

OS Version

14

ONNX Runtime Installation

Built from Source

Compiler Version (if 'Built from Source')

No response

Package Name (if 'Released Package')

None

ONNX Runtime Version or Commit ID

2a38168f0b491123470a30b473ac716eaa637656

ONNX Runtime API

C++/C

Architecture

X64

Execution Provider

SNPE

Execution Provider Library Version

2.18.0.240101

[github-actions[bot]]

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[kthur]

Did you push the qnn libraries to the device? The onnxruntime needs the qnn library to execute models.

[bibekanandaburagohain]

no, I haven't pushed anything. I just added the libQnnHtp.so in jnilibs folder in android. Can you share a little more details about which libraries are needed and what would be the absolute path to push these in?

[HectorSVC]

https://docs.qualcomm.com/bundle/publicresource/topics/80-63442-50/backend.html

[June1124]

Did you push the qnn libraries to the device? The onnxruntime needs the qnn library to execute models.

Hi, under which path of the device does the library need to be placed?

[DakeQQ]

I used the following official demo code to push the library, but encountered the error with HTP backend Android 8Gen2 device. "QNN SetupBackend failed: Failed to create device. Error: 14001". std::vector<const char> options_keys = {"backend_path"}; std::vector<const char> options_values = {"/data/user/0/com.example.myapplication/cache/libQnnHtp.so"};

However, the official demo run_qnn_ep_sample.bat file includes multiple libraries:

1. libQnnHtp.so
2. libQnnHtpV73Stub.so
3. libQnnHtpV73Skel.so
4. libQnnHtpPrepare.so
5. libqnnhtpv73.cat
6. libQnnCpu.so
7. libQnnSystem.so

I also tried using options_values.push_back("/data/user/0/com.example.myapplication/cache/.so"), or copied these .so files to the cache folder (the same folder as libQnnHtp.so), but it still failed.

How can I provide these libraries to the device to successfully run the HTP backend on Android device?

Build Info:

• libonnxruntime.so = 1.18
• QNN SDK = 2.22*

[kthur]

I used the following official demo code to push the library, but encountered the error with HTP backend Android 8Gen2 device. "QNN SetupBackend failed: Failed to create device. Error: 14001". std::vector<const char> options_keys = {"backend_path"}; std::vector<const char> options_values = {"/data/user/0/com.example.myapplication/cache/libQnnHtp.so"};

However, the official demo run_qnn_ep_sample.bat file includes multiple libraries:

1. libQnnHtp.so

2. libQnnHtpV73Stub.so

3. libQnnHtpV73Skel.so

4. libQnnHtpPrepare.so

5. libqnnhtpv73.cat

6. libQnnCpu.so

7. libQnnSystem.so

I also tried using options_values.push_back("/data/user/0/com.example.myapplication/cache/.so"), or copied these .so files to the cache folder (the same folder as libQnnHtp.so), but it still failed.

How can I provide these libraries to the device to successfully run the HTP backend on Android device?

Build Info:

* libonnxruntime.so = 1.18

* QNN SDK = 2.22*

I just export the library path using the LD_LIBRARY_PATH. Linux(android) occurs the fault without the libary, they are searching some folders. such as /vendor/lib64, /sytem/lib64 .... It depends on the systems. As I set the LD_LIBARY_PATH, it also search the library in LD_LIBRARY_PATH.

$ export LD_LIBRARY_PATH="/data/user/0/com.example.myapplication/cache/"