search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.79k stars 2.94k forks source link

[Mobile] QNN Execution provider mem alloc failure. #22288

Open iknowican opened 1 month ago

iknowican commented 1 month ago

Describe the issue

I have Snapdragon 8 Gen 1+ (v69 platform) and i am trying to initialize ONNX runtime with QNN backend (with NPU support).

I followed the instruction described if the official documentation from QNN execution provider and got onnxruntime.so for arm64-v8a. Doc states, that i need to put QNN dependencies such as libQnnHtp.so inside my .apk.

After this step i got following list of libraries (from Qualcomm AI SDK):

Here is a code, that i use to initialize session:

    auto file = fs.open("assets/models/convit_tiny_Opset16.onnx");
    auto model = std::string_view(std::cbegin(file), std::cend(file));
    Ort::Env env = Ort::Env{ORT_LOGGING_LEVEL_VERBOSE, "Default"};
    std::unordered_map<std::string, std::string> qnn_options;
    qnn_options["backend_path"] = "libQnnHtp.so";
    //qnn_options["enable_htp_fp16_precision"] = "1";
    //qnn_options["soc_model"] = "42"; //Current SoC model id
    Ort::SessionOptions session_options;
    session_options.AppendExecutionProvider("QNN", qnn_options);
    Ort::Session session(env, model.data(), model.size(), session_options);

I used a model from ONNX model zoo, small enough, to check that problem persists on any model.

At this point i got the following output:

I onnxruntime:  [I:onnxruntime:, inference_session.cc:583 TraceSessionOptions] Session Options {  execution_mode:0 execution_order:DEFAULT enable_profiling:0 optimized_model_filepath:"" enable_mem_pattern:1 enable_mem_reuse:1 enable_cpu_mem_arena:1 profile_file_prefix:onnxruntime_profile_ session_logid: session_log_severity_level:-1 session_log_verbosity_level:0 max_num_graph_transformation_steps:10 graph_optimization_level:3 intra_op_param:OrtThreadPoolParams { thread_pool_size: 0 auto_set_affinity: 0 allow_spinning: 1 dynamic_block_base_: 0 stack_size: 0 affinity_str:  set_denormal_as_zero: 0 } inter_op_param:OrtThreadPoolParams { thread_pool_size: 0 auto_set_affinity: 0 allow_spinning: 1 dynamic_block_base_: 0 stack_size: 0 affinity_str:  set_denormal_as_zero: 0 } use_per_session_threads:1 thread_pool_allow_spinning:1 use_deterministic_compute:0 config_options: {   QNN:backend_path: libQnnHtp.so } }
I onnxruntime:  [I:onnxruntime:, inference_session.cc:483 operator()] Flush-to-zero and denormal-as-zero are off
I onnxruntime:  [I:onnxruntime:, inference_session.cc:491 ConstructorCommon] Creating and using per session threadpools since use_per_session_threads_ is true
I onnxruntime:  [I:onnxruntime:, inference_session.cc:509 ConstructorCommon] Dynamic block base set to 0
D ViewRootImpl[QtActivity]: ViewUI_blast Received frameCompleteCallback  lastAcquiredFrameNum=1 lastAttemptedDrawFrameNum=1
I HwViewRootImpl: removeInvalidNode jank list is null
I HwViewRootImpl: remove sceneId 1 topId: 0
I HwViewRootImpl: Add sceneId 10 topId: 0
I ViewRootImpl[QtActivity]: ViewUI reportDrawFinished
D ActionMenuPresenter: initForMenu: width = mWidthLimit = 606
D ActionMenuPresenter: initForMenu: width -= mHwOverflowButton.getMeasuredWidth() width: 481
D ActionMenuPresenter: initForMenu: mActionItemWidthLimit = width: 481
D ActionMenuPresenter: initForMenu:  MIN_CELL_SIZE: 56 density: 3.125
D ActionMenuPresenter: initForMenu: width = mWidthLimit = 606
D ActionMenuPresenter: initForMenu: width -= mHwOverflowButton.getMeasuredWidth() width: 481
D ActionMenuPresenter: initForMenu: mActionItemWidthLimit = width: 481
D ActionMenuPresenter: initForMenu:  MIN_CELL_SIZE: 56 density: 3.125
I InsetsSourceConsumer: ViewUI_insets_ime removeImeSurface window: android.view.ViewRootImpl$W@2b6eeda
D InsetsSourceConsumer: ViewUI_insets setControl -> ITYPE_NAVIGATION_BAR on org.qtproject.example.qt_mobile/org.qtproject.qt.android.bindings.QtActivity
D InsetsSourceConsumer: ViewUI_insets setControl mType:1, requestedVisible:true, getLeash:Surface(name=Surface(name=ef8595b NavigationBar0)/@0x7118c10 - animation-leash of insets_animation)/@0xb464f0b, needAnimation:false, mIsAnimationPending:false, getRootViewTitle:org.qtproject.example.qt_mobile/org.qtproject.qt.android.bindings.QtActivity
D InsetsSourceConsumer: ViewUI_insets setControl -> ITYPE_STATUS_BAR on org.qtproject.example.qt_mobile/org.qtproject.qt.android.bindings.QtActivity
D InsetsSourceConsumer: ViewUI_insets setControl mType:0, requestedVisible:true, getLeash:Surface(name=Surface(name=663d03b StatusBar)/@0x4e2fde7 - animation-leash of insets_animation)/@0x68731e8, needAnimation:false, mIsAnimationPending:false, getRootViewTitle:org.qtproject.example.qt_mobile/org.qtproject.qt.android.bindings.QtActivity
I ActivityManager_activity: Reporting idle of ActivityRecord{4756093 token=android.os.BinderProxy@bcd8079 {org.qtproject.example.qt_mobile/org.qtproject.qt.android.bindings.QtActivity}} finished=false
I ViewRootImpl[QtActivity]: ViewUI windowFocusChanged: hasFocus true inTouchMode true
D HwViewRootImpl: [DetectViewsLocationRunner] updateResolution
D HwViewRootImpl: [DetectViewsLocationRunner] current resolutionScale: 1.0
I ViewRootImpl[QtActivity]: ViewUI handleWindowFocusChanged: hasWindowFocus true inTouchMode true
I ImeFocusController: ViewUI_insets_ime onPreWindowFocus, set InputMethodManager#mCurRootView. mViewRootImpl is android.view.ViewRootImpl@a352001
I ImeFocusController: ViewUI imeDisplayId:0 contextDisplayId:0
I InputMethodManager: set currentRootView to : android.view.ViewRootImpl@a352001
D ImeFocusController: ViewUI_insets_ime onViewFocusChanged, view=DecorView@a251ea6[QtActivity], mServedView=null hasFocus: true
V ImeFocusController: ViewUI_insets_ime checkFocus: view=null next=DecorView@a251ea6[QtActivity] force=true package=<none>
I InputMethodManager: Starting input: view=DecorView@a251ea6[QtActivity],focus=false,windowFocus=true,autofillUiShowing=false,window=android.view.ViewRootImpl$W@2b6eeda,displayId=0,temporaryDetach=false,hasImeFocus=true reason=WINDOW_FOCUS_GAIN
I InputMethodManager: Starting input: ic=null
W HwRemoteInputMethodManager: isCasting false because IHwDistributedWindowManager is invalid.
I InputMethodManager: START INPUT: startInputFlags=VIEW_HAS_FOCUS|INITIAL_CONNECTION
I InputMethodManager: Starting input: Bind resultString=SUCCESS_WAITING_IME_SESSION
I InputMethodManager: handleMessage: MSG_SET_ACTIVE true, was false
I InputMethodManager: handleMessage: MSG_BIND 602,com.huawei.ohos.inputmethod/com.android.inputmethod.latin.LatinIME
I InputMethodManager: Starting input: view=DecorView@a251ea6[QtActivity],focus=false,windowFocus=true,autofillUiShowing=false,window=android.view.ViewRootImpl$W@2b6eeda,displayId=0,temporaryDetach=false,hasImeFocus=true reason=BOUND_TO_IMMS
I InputMethodManager: Starting input: ic=null
I InputMethodManager: Starting input: finished by someone else. servedView=DecorView@a251ea6[QtActivity],focus=false,windowFocus=true,autofillUiShowing=false,window=android.view.ViewRootImpl$W@2b6eeda,displayId=0,temporaryDetach=false,hasImeFocus=true mServedConnecting=false
D InsetsSourceConsumer: ViewUI_insets setControl -> ITYPE_IME on org.qtproject.example.qt_mobile/org.qtproject.qt.android.bindings.QtActivity
I InsetsSourceConsumer: ViewUI_insets_ime removeImeSurface window: android.view.ViewRootImpl$W@2b6eeda
I ample.qt_mobil: jit_compiled:[OK] int android.icu.impl.ICUBinary.compareKeys(java.lang.CharSequence, byte[], int) @ /apex/com.android.i18n/javalib/core-icu4j.jar
V onnxruntime:  [V:onnxruntime:Default, qnn_execution_provider.cc:187 QNNExecutionProvider] Context cache enable: 0
V onnxruntime:  [V:onnxruntime:Default, qnn_execution_provider.cc:198 QNNExecutionProvider] User specified context cache embed mode: 1
V onnxruntime:  [V:onnxruntime:Default, qnn_execution_provider.cc:201 QNNExecutionProvider] User specified context cache path:
V onnxruntime:  [V:onnxruntime:Default, qnn_execution_provider.cc:208 QNNExecutionProvider] User specified QNN context node name prefix:
V onnxruntime:  [V:onnxruntime:Default, qnn_execution_provider.cc:212 QNNExecutionProvider] User specified option - share EP contexts across sessions: 0
V onnxruntime:  [V:onnxruntime:Default, qnn_execution_provider.cc:221 QNNExecutionProvider] Backend path: libQnnHtp.so
V onnxruntime:  [V:onnxruntime:Default, qnn_execution_provider.cc:297 QNNExecutionProvider] Profiling file path:
I onnxruntime:  [I:onnxruntime:, inference_session.cc:1669 Initialize] Initializing session.
I onnxruntime:  [I:onnxruntime:, inference_session.cc:1706 Initialize] Adding default CPU execution provider.
I onnxruntime:  [I:onnxruntime:Default, bfc_arena.cc:29 BFCArena] Creating BFCArena for Cpu with following configs: initial_chunk_size_bytes: 1048576 max_dead_bytes_per_chunk: 134217728 initial_growth_chunk_size_bytes: 2097152 max_power_of_two_extend_bytes: 1073741824 memory limit: 18446744073709551615 arena_extend_strategy: 0
V onnxruntime:  [V:onnxruntime:Default, bfc_arena.cc:66 BFCArena] Creating 21 bins of max chunk size 256 to 268435456
W libc    : Access denied finding property "ro.hardware.chipname"
I onnxruntime:  [I:onnxruntime:, graph_partitioner.cc:898 InlineFunctionsAOT] This model does not have any local functions defined. AOT Inlining is not performed
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer EnsureUniqueDQForNodeUnit modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer Level1_RuleBasedTransformer modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer DoubleQDQPairsRemover modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, constant_sharing.cc:248 ApplyImpl] Total shared scalar initializer count: 178
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer ConstantSharing modified: 1 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer CommonSubexpressionElimination modified: 1 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer ConstantFolding modified: 1 with status: OK
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/Equal_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/Mul_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.9/attn/pos_proj/Add_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.9/attn/pos_proj/MatMul_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.8/attn/Softmax_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.7/attn/pos_proj/MatMul_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.7/attn/Sigmoid_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/Where_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.6/attn/Transpose_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.6/attn/pos_proj/Add_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.9/attn/Sigmoid_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.6/attn/pos_proj/MatMul_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.6/attn/Sigmoid_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.5/attn/Softmax_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.5/attn/Transpose_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.4/attn/pos_proj/Add_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.4/attn/pos_proj/MatMul_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.3/attn/Softmax_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.3/attn/Transpose_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.3/attn/pos_proj/Add_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.3/attn/pos_proj/MatMul_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.3/attn/Sigmoid_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.4/attn/Sigmoid_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.2/attn/Transpose_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.2/attn/Sigmoid_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.5/attn/pos_proj/Add_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/ConstantOfShape_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.1/attn/Softmax_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.1/attn/Transpose_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.1/attn/pos_proj/Add_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.1/attn/pos_proj/MatMul_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.0/attn/Transpose_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.0/attn/pos_proj/Add_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.0/attn/Where_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.0/attn/ConstantOfShape_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.0/attn/Sigmoid_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.7/attn/Softmax_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.9/attn/Constant_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.8/attn/Constant_3_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.2/attn/Softmax_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.0/attn/Mul_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.5/attn/Constant_2_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.5/attn/pos_proj/MatMul_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.0/attn/Constant_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::MatMul_1671'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/Constant_2_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::MatMul_1932'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'blocks.8.attn.pos_proj.bias'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Sigmoid_1730'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::MatMul_1903'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Sigmoid_1875'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::MatMul_1874'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Sigmoid_1846'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::MatMul_1845'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.2/attn/pos_proj/Add_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.4/attn/Transpose_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Sigmoid_1788'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/patch_embed/Shape_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Sigmoid_1701'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::MatMul_1729'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::MatMul_1700'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Sigmoid_1672'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/patch_embed/Slice_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.1/attn/Sigmoid_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'blocks.1.attn.pos_proj.bias'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.0/attn/Equal_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'blocks.9.attn.pos_proj.bias'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.8/attn/pos_proj/MatMul_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::MatMul_1758'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'blocks.5.attn.pos_proj.bias'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.4/attn/Softmax_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.9/attn/Softmax_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/Constant_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'cls_token'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.8/attn/pos_proj/Add_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'blocks.7.attn.pos_proj.bias'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.7/attn/pos_proj/Add_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.5/attn/Sigmoid_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.0/attn/pos_proj/MatMul_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.9/attn/Transpose_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.0/attn/Softmax_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'blocks.6.attn.pos_proj.bias'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Sigmoid_1759'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'blocks.0.attn.pos_proj.bias'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.7/attn/Transpose_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.0/attn/Expand_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'blocks.3.attn.pos_proj.bias'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'blocks.4.attn.pos_proj.bias'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.8/attn/Transpose_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Sigmoid_1817'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Sigmoid_1933'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Sigmoid_1904'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'blocks.2.attn.pos_proj.bias'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.8/attn/Sigmoid_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.6/attn/Softmax_1_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/patch_embed/Constant_2_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::MatMul_1816'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer '/blocks.2/attn/pos_proj/MatMul_output_0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4288 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::MatMul_1787'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer MatMulAddFusion modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer ReshapeFusion modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer FreeDimensionOverrideTransformer modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer GeluFusionL1 modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer LayerNormFusionL1 modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer QDQPropagationTransformer modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer EnsureUniqueDQForNodeUnit modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer RocmBlasAltImpl modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer TransposeOptimizer modified: 1 with status: OK
I onnxruntime:  [I:onnxruntime:, graph.cc:4351 CleanUnusedInitializersAndNodeArgs] Removing NodeArg 'Transpose_token_8_out0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4351 CleanUnusedInitializersAndNodeArgs] Removing NodeArg 'Transpose_token_6_out0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4351 CleanUnusedInitializersAndNodeArgs] Removing NodeArg 'Transpose_token_5_out0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4351 CleanUnusedInitializersAndNodeArgs] Removing NodeArg 'Transpose_token_2_out0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4351 CleanUnusedInitializersAndNodeArgs] Removing NodeArg 'Transpose_token_0_out0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4351 CleanUnusedInitializersAndNodeArgs] Removing NodeArg 'Transpose_out0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4351 CleanUnusedInitializersAndNodeArgs] Removing NodeArg 'Transpose_token_11_out0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph.cc:4351 CleanUnusedInitializersAndNodeArgs] Removing NodeArg 'Transpose_token_4_out0'. It is no longer used by any node.
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer Level1_RuleBasedTransformer modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer DoubleQDQPairsRemover modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer CommonSubexpressionElimination modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer ConstantFolding modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer MatMulAddFusion modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer ReshapeFusion modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer FreeDimensionOverrideTransformer modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer GeluFusionL1 modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer LayerNormFusionL1 modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer QDQPropagationTransformer modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer EnsureUniqueDQForNodeUnit modified: 0 with status: OK
I onnxruntime:  [I:onnxruntime:, graph_transformer.cc:15 Apply] GraphTransformer RocmBlasAltImpl modified: 0 with status: OK
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:75 GetQnnInterfaceProvider] libQnnHtp.so interface version: 2.19.0
I onnxruntime:  [I:onnxruntime:Default, qnn_backend_manager.cc:134 LoadBackend] Found valid interface, version: 2.19.0 backend provider name: HTP_QTI_AISW backend id: 6
V onnxruntime:  [V:onnxruntime:, qnn_backend_manager.cc:700 SetupBackend] LoadBackend succeed.
V onnxruntime:  [V:onnxruntime:, qnn_backend_manager.cc:707 SetupBackend] Backend build version: v2.26.0.240827110523_99241
V onnxruntime:  [V:onnxruntime:, qnn_backend_manager.cc:260 InitializeQnnLog] Set Qnn log level: 5
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <I> QnnLog_create started.
08-04 13:56:51.506 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <W> Initializing HtpProvider
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Creating default router
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> RouterFastRPC create
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: Initializing the router
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <I> exit with 0
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <I> exit with 0
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <I> First connection to QNN stub established!
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Loading remote funcs
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Async property not supported. Skipping setup async threads
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Detected Snapdragon SOC SM8475 with 3 SOCs
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Allocating PlatformInfo struct size 120
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: Initializing the log registry
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: Initializing the graph registry
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: Initializing the context registry
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: Initializing the device registry
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: Initializing the tensor counter
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: setting isExitCalled to false
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: initializing mem registry
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: initializing mmap registry
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: initializing graphId registry
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: initializing debugFactory
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: constructing bundle
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> RouterFastRPC isPrepareLibLoaded
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP: initialization completed successfully
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <I> Applying log level 5 to 0 devices
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> RouterFastRPC initializeNativeOpValidatorLogs
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <W> Function not called, PrepareLib isn't loaded!
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> HTP Prepare backend not loaded
08-04 13:56:51.507 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <I> QnnLog_create exit.
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:, qnn_backend_manager.cc:711 SetupBackend] SetLogger succeed.
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <I> QnnBackend_create started. backend = 0x85437640
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Backend handle created: 1
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> RouterFastRPC isPrepareLibLoaded
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <I> QnnBackend_create done successfully. backend = 0x85437640
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:, qnn_backend_manager.cc:714 SetupBackend] InitializeBackend succeed.
I onnxruntime:  [I:onnxruntime:Default, qnn_backend_manager.cc:409 CreateDevice] Create device.
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <I> QnnDevice_create started
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Create device with id 0x1
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Config not passed. Loading default config
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Setting default value for unsigned PD usage
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> skel file path file:///libQnnHtpV69Skel.so?qnn_skel_handle_invoke&_modver=1.0&_dom=cdsp
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Transport session for deviceId 0 coreId 0 pdId 0 not found!
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> DeviceId 0 coreId 0 pdId 0 not present, insert a new entry 0xb400007c7f1e6318
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <I> exits with 2, successfully initialized rpc memory
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Successful rpcMemInit
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <I> rpcMemoryAlloc 8 isInit 1
08-04 13:56:51.508 20286 20355 V onnxruntime:
E adsprpc : remote.c:73:rpcmem alloc stubbed routine - Return failure
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <E> Failed to allocate memory
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Found transport session 0xb400007c7f1e6318 for deviceId 0 coreId 0 pdId 0!
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> Transport layer for DeviceId 0 coreId 0 pdId 0 is now Removed
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <E> Failed to create transport for device, error: 4000
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <E> Failed to load skel, error: 4000
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <V> exits device initialization with  14001
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <E> Transport layer setup failed: 14001
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <E> Failed to parse default platform info: 14001
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <E> Failed to load default platform info: 14001
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <E> Failed to parse platform config: 14001
08-04 13:56:51.508 20286 20355 V onnxruntime:
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <I> QnnDevice_create done. status 0x36b1
08-04 13:56:51.508 20286 20355 V onnxruntime:
E onnxruntime:  [E:onnxruntime:, qnn_execution_provider.cc:662 GetCapability] QNN SetupBackend failed Failed to create device. Error: QNN_DEVICE_ERROR_INVALID_CONFIG: Invalid config values

As i can see, the main problem is here

E adsprpc : remote.c:73:rpcmem alloc stubbed routine - Return failure
V onnxruntime:  [V:onnxruntime:Default, qnn_backend_manager.cc:251 QnnLogging]  <E> Failed to allocate memory

So, it looks like i picked right dependencies, and, at least, QNN EP is constructed, but, for some unknown reason, failed to allocate memory on DSP. I guess this problem can be related to Qualcomm AI SDK or CPU/NPU itself, but here is a chance, that QNN EP may handle something wrong (or i am missing something).

To reproduce

Provided in description.

In general, build QNN EP according to official instructions with standard Qualcomm libraries (i used QAIRT 2.27.0.240926 and Hexagon SDK 6.0.0.2), then try to initialize session using QNN EP with default settings and any model on Snapdragon 8 Gen 1+.

Urgency

N/A

Platform

Android

OS Version

API 29, 30

ONNX Runtime Installation

Built from Source

Compiler Version (if 'Built from Source')

Clang 17.0.2

Package Name (if 'Released Package')

onnxruntime-mobile

ONNX Runtime Version or Commit ID

a47254eaef87a4572078d3165f4b9d1c4b4c52b1

ONNX Runtime API

C++/C

Architecture

ARM64

Execution Provider

Other / Unknown

Execution Provider Library Version

QNN

HectorSVC commented 1 month ago

more like environment issue, make sure you have export VENDOR_LIB=/vendor/lib64/ export LD_LIBRARY_PATH=/data/local/tmp/your_qnn/v227:/vendor/dsp/cdsp:$VENDOR_LIB export ADSP_LIBRARY_PATH="/data/local/tmp/qnn/v227;/vendor/dsp/cdsp;/vendor/lib/rfsa/adsp;/system/lib/rfsa/adsp;/dsp" similar issue to https://github.com/microsoft/onnxruntime/issues/21887

edgchen1 commented 1 month ago

try setting ADSP_LIBRARY_PATH so that it contains the directory with the QNN libs

sushuChina commented 1 month ago

Whether to use setting ADSP_LIBRARY_PATH when building onnxrntime

sushuChina commented 1 month ago

Have you solved this problem? I also encountered the same error 2024-10-08_18-07

HectorSVC commented 1 month ago

Snapdragon 8 Gen 1 is v69, make sure you copied the v69 libs from hexagon-v69 folder to the folder which has libQnnHtp.so

github-actions[bot] commented 1 week ago

This issue has been automatically marked as stale due to inactivity and will be closed in 30 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

iknowican commented 1 week ago

Well, it took a bit of time to figure out what was wrong in my case.

First, i use Qt toolchain, so, it was crucial to set up Qt/CMake variable to import Android manifest with explicitly declared native libraries (libcdsprpc.so). I missed this step and got auto-generated manifest without platform libs declaration. It solved the problem, but partially.

Second, right after resolving issue with libcdsprpc.so, i got errors about access to tmpfs. At this point everything turned into a battle versus SELinux and apk restrictions. I spent a few days on it and switched to another chip without such problems.

That's all for now. Thanks a lot!