search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.17k stars 2.86k forks source link

React Native .ort Operator Issue #12743

Closed jackylu0124 closed 1 year ago

jackylu0124 commented 2 years ago

Describe the bug

I have been trying to create inference sessions on Android with .ort models that contain operators like "ReduceMax", "NonZero", "Expand", "Where", "ReduceSum", etc, but the inference session creation failed with the message [Error: Can't load a model: Can't create InferenceSession]. When I printed out OrtException's message by adding print statement in the loadModel function in OnnxruntimeModule.java, I see, for example in the case of the ReduceSum operator, the error message Error code - ORT_FAIL - message: session_state.cc:1015 operator() Failed to find kernel def hash (9402732746386583360) in kernel registries for ReduceSum(13) node with name 'ReduceSum_5'. I have checked the list of operators supported by ORT Mobile at https://onnxruntime.ai/docs/reference/operators/mobile_package_op_type_support_1.11.html, and the aforementioned operators are all included in the list.

For the sake of reproduction of the issue, I have created a very simple model that involves only the ReduceSum operator, and the model (see screenshot below, the model takes in an int32 tensor and counts the number of entries that are equal to the number 3 in the tensor) can reproduce the issue. I have also included the code that exports the ONNX model from PyTorch code, the script command I used to convert the .onnx file to .ort file, as well as the React Native projects that try to create inference session with the model in this repo: https://github.com/jackylu0124/ort-model-issue

Urgency

High, this is a blocking issue to my project.

System information

To Reproduce

Link to repo with models and minimal code for error reproduction: https://github.com/jackylu0124/ort-model-issue Please follow instructions on https://reactnative.dev/docs/environment-setup in order to set up and run the project with Android Studio's emulator.

Once the project is set up and running, you can click/press the blue button on the top of the screen, which will try to create an inference session with the FindCountOfThree_v001.ort model, and the error message [Error: Can't load a model: Can't create InferenceSession] will be logged into the Metro console. In order to see the more detailed Java log message like Error code - ORT_FAIL - message: session_state.cc:1015 operator() Failed to find kernel def hash (9402732746386583360) in kernel registries for ReduceSum(13) node with name 'ReduceSum_5'., please see the messages logged inside the "Logcat" section in Android Studio.

Notes:

  1. I have created two React Native projects in which one passes in the .ort model's file path to create the inference session (see the ORTInference/ folder), and one passes in the Uint8Array (using the quick hack from here) to create the inference session (see the ORTInferenceWithBuffer/ folder). The reason I included both approaches is to confirm that both ways generate the same error message and that the error message is not caused by model parsing issue, which seems to cause some problems for other people.
  2. The onnx_model_export.ipynb notebook exports the onnx_models/FindCountOfThree_v001.onnx model from PyTorch module, and the commands listed in onnx_models_to_ort_conversion_script.txt are used to convert the .onnx file to FindCountOfThree_v001.optimized.onnx and FindCountOfThree_v001.ort under onnx_models_optimized/.
  3. The Uint8Array used to represent the model in the ORTInferenceWithBuffer/ project is created by parsign a hex string. The code for converting the .ort file to the .txt file containing the hex string can be found in onnx_hex_files/.
  4. I have tried setting the Android React Native project's minSdkVersion to 28 (which is the tested version according to https://onnxruntime.ai/docs/reference/compatibility.html) inside android/build.gradle, but the issue and the same error messages still persist.

Expected behavior

Inference sessions created with models containing operators like "ReduceMax", "NonZero", "Expand", "Where", "ReduceSum", etc. should succeed, as they are included in the list of supported ORT mobile operators on the documentation page https://onnxruntime.ai/docs/reference/operators/mobile_package_op_type_support_1.11.html.

Screenshots

Screenshot of the .ort model: image

Additional context Similar error messages that I have encountered in other models that use the operators "ReduceMax", "NonZero", "Expand", "Where":

  1. Error code - ORT_FAIL - message: session_state.cc:1015 operator() Failed to find kernel def hash (15302447573039984352) in kernel registries for ReduceMax(13) node with name 'ReduceMax_95'.
  2. Error code - ORT_FAIL - message: session_state.cc:1015 operator() Failed to find kernel def hash (15791942659448002792) in kernel registries for NonZero(13) node with name 'NonZero_127'.
  3. Error code - ORT_FAIL - message: session_state.cc:1015 operator() Failed to find kernel def hash (15910553252200910392) in kernel registries for Expand(13) node with name 'Expand_404'.
  4. Error code - ORT_FAIL - message: session_state.cc:1015 operator() Failed to find kernel def hash (18032063618220677104) in kernel registries for Where(9) node with name 'Where_267'.
edgchen1 commented 2 years ago

The ops listed in the mobile package only support a subset of data types: https://onnxruntime.ai/docs/reference/operators/mobile_package_op_type_support_1.11.html#supported-data-input-types

In this case, the ReduceSum int64 kernel is not included.

To work around it and still use the existing package, you could try casting to float prior to the ReduceSum.

Or you can do a custom build of ORT to include the kernels that you want. A config file with those ops and types should be generated from the ORT format conversion process. If you don't have binary size constraints, you can just do a full build for simplicity.

For the Android package, we do have both "mobile" (onnxruntime-mobile) and "full" (onnxruntime-android) versions, with the latter including a full build of ORT. I'm not sure if React Native packages have a version with a full build. @fs-eire do you know?

fs-eire commented 2 years ago

The ops listed in the mobile package only support a subset of data types: https://onnxruntime.ai/docs/reference/operators/mobile_package_op_type_support_1.11.html#supported-data-input-types

In this case, the ReduceSum int64 kernel is not included.

To work around it and still use the existing package, you could try casting to float prior to the ReduceSum.

Or you can do a custom build of ORT to include the kernels that you want. A config file with those ops and types should be generated from the ORT format conversion process. If you don't have binary size constraints, you can just do a full build for simplicity.

For the Android package, we do have both "mobile" (onnxruntime-mobile) and "full" (onnxruntime-android) versions, with the latter including a full build of ORT. I'm not sure if React Native packages have a version with a full build. @fs-eire do you know?

This is the embeded build.gradle file inside onnxruntime-react-native: https://github.com/microsoft/onnxruntime/blob/main/js/react_native/android/build.gradle#L135

Is there a way to know which version is it using?

jackylu0124 commented 2 years ago

Hey @edgchen1 and @fs-eire,

Thank you very much for the insight and apologies for the delay. Following your suggestions, I am currently following this part of the documentation (https://onnxruntime.ai/docs/install/#install-on-web-and-mobile, also shown in screenshot below) in order to use the full onnxruntime build for my React Native app. I have two questions:

  1. For step 1 in the "Install on Android section" on the documentation/instructions site, my repositories section in my React Native project's MyProject/android/build.gradle looks like the following:

    allprojects {
repositories {
    maven {
        // All of React Native (JS, Obj-C sources, Android binaries) is installed from npm
        url("$rootDir/../node_modules/react-native/android")
    }
    maven {
        // Android JSC is installed from npm
        url("$rootDir/../node_modules/jsc-android/dist")
    }
    mavenCentral {
        // We don't want to fetch react-native from Maven Central as there are
        // older versions over there.
        content {
            excludeGroup "com.facebook.react"
        }
    }
    google()
    maven { url 'https://www.jitpack.io' }
}
}

    ; in this case, how should I incorporate the 

    repositories {
 mavenCentral()
}

    change mentioned in step 1 in the "Install on Android section" on the documentation/instructions site?

  2. For the Android part specifically, the site says "Include the header files from the headers folder, and the relevant libonnxruntime.so dynamic library from the jni folder in your NDK project."; I am able download the onnxruntime-android AAR hosted at MavenCentral and was able to unzip it, but I wasn't sure where exactly in my React Native app project's folder structure I should put the header files from the headers folder and also the libonnxruntime.so dynamic library from the jni folder. Do you by chance have a sample React Native project that uses the full onnxruntime build that I could reference and take a look at? (For references, I have also pasted the current folder structure of MyProject/android/app/ in my React Native project down below)

  3. For the iOS part, do I need to download header files and dynamic libraries like Android from MavenCentral for example in order to install the full onnxruntime build for iOS. If so, where can I download them?

Thank you very much for all the help again!

P.S.

Instructions (https://onnxruntime.ai/docs/install/#install-on-web-and-mobile) screenshot

image

Current folder structure at MyReactNativeProject/android/app/ in my React Native project

+---build
|   +---generated
|   |   +---ap_generated_sources
|   |   |   \---debug
|   |   |       \---out
|   |   +---res
|   |   |   +---pngs
|   |   |   |   \---debug
|   |   |   \---resValues
|   |   |       \---debug
|   |   |           \---values
|   |   +---rncli
|   |   |   \---src
|   |   |       \---main
|   |   |           \---java
|   |   |               \---com
|   |   |                   \---facebook
|   |   |                       \---react
|   |   \---source
|   |       \---buildConfig
|   |           \---debug
|   |               \---com
|   |                   \---onnxmodeltestbed
|   +---intermediates
|   |   +---aar_metadata_check
|   |   |   \---debug
|   |   +---annotation_processor_list
|   |   |   \---debug
|   |   +---app_metadata
|   |   |   \---debug
|   |   +---compatible_screen_manifest
|   |   |   \---debug
|   |   +---compile_and_runtime_not_namespaced_r_class_jar
|   |   |   \---debug
|   |   +---compressed_assets
|   |   |   \---debug
|   |   |       \---out
|   |   +---data_binding_layout_info_type_merge
|   |   |   \---debug
|   |   |       \---out
|   |   +---desugar_graph
|   |   |   \---debug
|   |   |       \---out
|   |   |           +---currentProject
|   |   |           |   +---dirs_bucket_0
|   |   |           |   +---dirs_bucket_1
|   |   |           |   +---dirs_bucket_2
|   |   |           |   +---dirs_bucket_3
|   |   |           |   +---dirs_bucket_4
|   |   |           |   +---dirs_bucket_5
|   |   |           |   +---dirs_bucket_6
|   |   |           |   +---dirs_bucket_7
|   |   |           |   +---jar_aa4d3e88fc01cf2951e2abf942e5c709f94f2147eb575a74073e166c6e6854f7_bucket_0
|   |   |           |   +---jar_aa4d3e88fc01cf2951e2abf942e5c709f94f2147eb575a74073e166c6e6854f7_bucket_1
|   |   |           |   +---jar_aa4d3e88fc01cf2951e2abf942e5c709f94f2147eb575a74073e166c6e6854f7_bucket_2
|   |   |           |   +---jar_aa4d3e88fc01cf2951e2abf942e5c709f94f2147eb575a74073e166c6e6854f7_bucket_3
|   |   |           |   +---jar_aa4d3e88fc01cf2951e2abf942e5c709f94f2147eb575a74073e166c6e6854f7_bucket_4
|   |   |           |   +---jar_aa4d3e88fc01cf2951e2abf942e5c709f94f2147eb575a74073e166c6e6854f7_bucket_5
|   |   |           |   +---jar_aa4d3e88fc01cf2951e2abf942e5c709f94f2147eb575a74073e166c6e6854f7_bucket_6
|   |   |           |   \---jar_aa4d3e88fc01cf2951e2abf942e5c709f94f2147eb575a74073e166c6e6854f7_bucket_7
|   |   |           +---externalLibs
|   |   |           +---mixedScopes
|   |   |           \---otherProjects
|   |   +---dex
|   |   |   \---debug
|   |   |       +---mergeExtDexDebug
|   |   |       +---mergeLibDexDebug
|   |   |       |   +---0
|   |   |       |   +---1
|   |   |       |   +---10
|   |   |       |   +---11
|   |   |       |   +---12
|   |   |       |   +---13
|   |   |       |   +---14
|   |   |       |   +---15
|   |   |       |   +---2
|   |   |       |   +---3
|   |   |       |   +---4
|   |   |       |   +---5
|   |   |       |   +---6
|   |   |       |   +---7
|   |   |       |   +---8
|   |   |       |   \---9
|   |   |       \---mergeProjectDexDebug
|   |   |           +---0
|   |   |           +---1
|   |   |           +---10
|   |   |           +---11
|   |   |           +---12
|   |   |           +---13
|   |   |           +---14
|   |   |           +---15
|   |   |           +---2
|   |   |           +---3
|   |   |           +---4
|   |   |           +---5
|   |   |           +---6
|   |   |           +---7
|   |   |           +---8
|   |   |           \---9
|   |   +---dex_archive_input_jar_hashes
|   |   |   \---debug
|   |   +---dex_number_of_buckets_file
|   |   |   \---debug
|   |   +---duplicate_classes_check
|   |   |   \---debug
|   |   +---external_file_lib_dex_archives
|   |   |   \---debug
|   |   +---external_libs_dex_archive
|   |   |   \---debug
|   |   |       \---out
|   |   +---external_libs_dex_archive_with_artifact_transforms
|   |   |   \---debug
|   |   |       \---out
|   |   +---incremental
|   |   |   +---debug-mergeJavaRes
|   |   |   |   \---zip-cache
|   |   |   +---mergeDebugAssets
|   |   |   +---mergeDebugJniLibFolders
|   |   |   +---mergeDebugResources
|   |   |   |   +---merged.dir
|   |   |   |   |   +---values
|   |   |   |   |   +---values-af
|   |   |   |   |   +---values-am
|   |   |   |   |   +---values-ar
|   |   |   |   |   +---values-as
|   |   |   |   |   +---values-az
|   |   |   |   |   +---values-b+sr+Latn
|   |   |   |   |   +---values-be
|   |   |   |   |   +---values-bg
|   |   |   |   |   +---values-bn
|   |   |   |   |   +---values-bs
|   |   |   |   |   +---values-ca
|   |   |   |   |   +---values-cs
|   |   |   |   |   +---values-da
|   |   |   |   |   +---values-de
|   |   |   |   |   +---values-el
|   |   |   |   |   +---values-en-rAU
|   |   |   |   |   +---values-en-rCA
|   |   |   |   |   +---values-en-rGB
|   |   |   |   |   +---values-en-rIN
|   |   |   |   |   +---values-en-rXC
|   |   |   |   |   +---values-es
|   |   |   |   |   +---values-es-rUS
|   |   |   |   |   +---values-et
|   |   |   |   |   +---values-eu
|   |   |   |   |   +---values-fa
|   |   |   |   |   +---values-fi
|   |   |   |   |   +---values-fr
|   |   |   |   |   +---values-fr-rCA
|   |   |   |   |   +---values-gl
|   |   |   |   |   +---values-gu
|   |   |   |   |   +---values-h720dp-v13
|   |   |   |   |   +---values-hdpi-v4
|   |   |   |   |   +---values-hi
|   |   |   |   |   +---values-hr
|   |   |   |   |   +---values-hu
|   |   |   |   |   +---values-hy
|   |   |   |   |   +---values-in
|   |   |   |   |   +---values-is
|   |   |   |   |   +---values-it
|   |   |   |   |   +---values-iw
|   |   |   |   |   +---values-ja
|   |   |   |   |   +---values-ka
|   |   |   |   |   +---values-kk
|   |   |   |   |   +---values-km
|   |   |   |   |   +---values-kn
|   |   |   |   |   +---values-ko
|   |   |   |   |   +---values-ky
|   |   |   |   |   +---values-land
|   |   |   |   |   +---values-large-v4
|   |   |   |   |   +---values-ldltr-v21
|   |   |   |   |   +---values-lo
|   |   |   |   |   +---values-lt
|   |   |   |   |   +---values-lv
|   |   |   |   |   +---values-mk
|   |   |   |   |   +---values-ml
|   |   |   |   |   +---values-mn
|   |   |   |   |   +---values-mr
|   |   |   |   |   +---values-ms
|   |   |   |   |   +---values-my
|   |   |   |   |   +---values-nb
|   |   |   |   |   +---values-ne
|   |   |   |   |   +---values-night-v8
|   |   |   |   |   +---values-nl
|   |   |   |   |   +---values-or
|   |   |   |   |   +---values-pa
|   |   |   |   |   +---values-pl
|   |   |   |   |   +---values-port
|   |   |   |   |   +---values-pt
|   |   |   |   |   +---values-pt-rBR
|   |   |   |   |   +---values-pt-rPT
|   |   |   |   |   +---values-ro
|   |   |   |   |   +---values-ru
|   |   |   |   |   +---values-si
|   |   |   |   |   +---values-sk
|   |   |   |   |   +---values-sl
|   |   |   |   |   +---values-sq
|   |   |   |   |   +---values-sr
|   |   |   |   |   +---values-sv
|   |   |   |   |   +---values-sw
|   |   |   |   |   +---values-sw600dp-v13
|   |   |   |   |   +---values-ta
|   |   |   |   |   +---values-te
|   |   |   |   |   +---values-th
|   |   |   |   |   +---values-tl
|   |   |   |   |   +---values-tr
|   |   |   |   |   +---values-uk
|   |   |   |   |   +---values-ur
|   |   |   |   |   +---values-uz
|   |   |   |   |   +---values-v16
|   |   |   |   |   +---values-v17
|   |   |   |   |   +---values-v18
|   |   |   |   |   +---values-v21
|   |   |   |   |   +---values-v22
|   |   |   |   |   +---values-v23
|   |   |   |   |   +---values-v24
|   |   |   |   |   +---values-v25
|   |   |   |   |   +---values-v26
|   |   |   |   |   +---values-v28
|   |   |   |   |   +---values-vi
|   |   |   |   |   +---values-watch-v20
|   |   |   |   |   +---values-watch-v21
|   |   |   |   |   +---values-xlarge-v4
|   |   |   |   |   +---values-zh-rCN
|   |   |   |   |   +---values-zh-rHK
|   |   |   |   |   +---values-zh-rTW
|   |   |   |   |   \---values-zu
|   |   |   |   \---stripped.dir
|   |   |   +---mergeDebugShaders
|   |   |   +---packageDebug
|   |   |   |   \---tmp
|   |   |   |       \---debug
|   |   |   |           \---zip-cache
|   |   |   \---processDebugResources
|   |   +---javac
|   |   |   \---debug
|   |   |       \---classes
|   |   |           \---com
|   |   |               +---facebook
|   |   |               |   \---react
|   |   |               \---onnxmodeltestbed
|   |   |                   \---newarchitecture
|   |   |                       +---components
|   |   |                       \---modules
|   |   +---manifest_merge_blame_file
|   |   |   \---debug
|   |   +---merged_assets
|   |   |   \---debug
|   |   |       \---out
|   |   +---merged_java_res
|   |   |   \---debug
|   |   +---merged_jni_libs
|   |   |   \---debug
|   |   |       \---out
|   |   +---merged_manifest
|   |   |   \---debug
|   |   +---merged_manifests
|   |   |   \---debug
|   |   +---merged_native_libs
|   |   |   \---debug
|   |   |       \---out
|   |   |           \---lib
|   |   |               +---arm64-v8a
|   |   |               +---armeabi-v7a
|   |   |               +---x86
|   |   |               \---x86_64
|   |   +---merged_res
|   |   |   \---debug
|   |   +---merged_res_blame_folder
|   |   |   \---debug
|   |   |       \---out
|   |   |           +---multi-v2
|   |   |           \---single
|   |   +---merged_shaders
|   |   |   \---debug
|   |   |       \---out
|   |   +---mixed_scope_dex_archive
|   |   |   \---debug
|   |   |       \---out
|   |   +---navigation_json
|   |   |   \---debug
|   |   +---packaged_manifests
|   |   |   \---debug
|   |   +---processed_res
|   |   |   \---debug
|   |   |       \---out
|   |   +---project_dex_archive
|   |   |   \---debug
|   |   |       \---out
|   |   |           \---com
|   |   |               +---facebook
|   |   |               |   \---react
|   |   |               \---onnxmodeltestbed
|   |   |                   \---newarchitecture
|   |   |                       +---components
|   |   |                       \---modules
|   |   +---runtime_symbol_list
|   |   |   \---debug
|   |   +---signing_config_versions
|   |   |   \---debug
|   |   +---stable_resource_ids_file
|   |   |   \---debug
|   |   +---stripped_native_libs
|   |   |   \---debug
|   |   |       \---out
|   |   |           \---lib
|   |   |               +---arm64-v8a
|   |   |               +---armeabi-v7a
|   |   |               +---x86
|   |   |               \---x86_64
|   |   +---sub_project_dex_archive
|   |   |   \---debug
|   |   |       \---out
|   |   +---symbol_list_with_package_name
|   |   |   \---debug
|   |   \---validate_signing_config
|   |       \---debug
|   +---outputs
|   |   +---apk
|   |   |   \---debug
|   |   \---logs
|   \---tmp
|       \---compileDebugJavaWithJavac
\---src
    +---debug
    |   \---java
    |       \---com
    |           \---onnxmodeltestbed
    \---main
        +---java
        |   \---com
        |       \---onnxmodeltestbed
        |           \---newarchitecture
        |               +---components
        |               \---modules
        +---jni
        \---res
            +---drawable
            +---mipmap-hdpi
            +---mipmap-mdpi
            +---mipmap-xhdpi
            +---mipmap-xxhdpi
            +---mipmap-xxxhdpi
            \---values
edgchen1 commented 2 years ago

This is the embeded build.gradle file inside onnxruntime-react-native: https://github.com/microsoft/onnxruntime/blob/main/js/react_native/android/build.gradle#L135

Is there a way to know which version is it using?

com.microsoft.onnxruntime:onnxruntime-mobile is specified and that is the mobile version. com.microsoft.onnxruntime:onnxruntime-android is the full version.

edgchen1 commented 2 years ago
  1. For step 1 in the "Install on Android section" on the documentation/instructions site, my repositories section in my React Native project's MyProject/android/build.gradle looks like the following:
allprojects {
    repositories {
        maven {
            // All of React Native (JS, Obj-C sources, Android binaries) is installed from npm
            url("$rootDir/../node_modules/react-native/android")
        }
        maven {
            // Android JSC is installed from npm
            url("$rootDir/../node_modules/jsc-android/dist")
        }
        mavenCentral {
            // We don't want to fetch react-native from Maven Central as there are
            // older versions over there.
            content {
                excludeGroup "com.facebook.react"
            }
        }
        google()
        maven { url 'https://www.jitpack.io' }
    }
}

; in this case, how should I incorporate the

repositories {
     mavenCentral()
}

change mentioned in step 1 in the "Install on Android section" on the documentation/instructions site?

These instructions are geared towards direct usage of the onnxruntime Android package. I'm not too familiar with React Native. However, if you can find and modify the embedded build.gradle file @fs-eire mentioned above (https://github.com/microsoft/onnxruntime/blob/main/js/react_native/android/build.gradle#L135), maybe you could change it to depend on com.microsoft.onnxruntime:onnxruntime-android instead. @fs-eire would that work?

As a side note, we will look into providing a React Native package with a full build in the future.

  1. For the Android part specifically, the site says "Include the header files from the headers folder, and the relevant libonnxruntime.so dynamic library from the jni folder in your NDK project."; I am able download the onnxruntime-android AAR hosted at MavenCentral and was able to unzip it, but I wasn't sure where exactly in my React Native app project's folder structure I should put the header files from the headers folder and also the libonnxruntime.so dynamic library from the jni folder. Do you by chance have a sample React Native project that uses the full onnxruntime build that I could reference and take a look at? (For references, I have also pasted the current folder structure of MyProject/android/app/ in my React Native project down below)

This is specific to using ORT from C/C++, which I believe is not what you are trying to do. Let us know if that's not the case.

  1. For the iOS part, do I need to download header files and dynamic libraries like Android from MavenCentral for example in order to install the full onnxruntime build for iOS. If so, where can I download them?

Consuming the iOS CocoaPods directly should work. I think this line would need to be updated to refer to 'onnxruntime-c' (the full build package) instead: https://github.com/microsoft/onnxruntime/blob/d19955fd89a18dbe82bed4396c7f2923995095fc/js/react_native/ios/Podfile#L17

jackylu0124 commented 2 years ago

Hi @edgchen1,

Thank you very much for your detailed reply and clarification! Please see the following for my replies:

These instructions are geared towards direct usage of the onnxruntime Android package. I'm not too familiar with React Native. However, if you can find and modify the embedded build.gradle file @fs-eire mentioned above (https://github.com/microsoft/onnxruntime/blob/main/js/react_native/android/build.gradle#L135), maybe you could change it to depend on com.microsoft.onnxruntime:onnxruntime-android instead. @fs-eire would that work?

I see, thanks for the clarification! For my Android project, I tried changing the line implementation "com.microsoft.onnxruntime:onnxruntime-mobile:latest.integration@aar" to implementation "com.microsoft.onnxruntime:onnxruntime-android" in the MyProject/node_modules/onnxruntime-react-native/android/build.gradle file, but when I tried to run the program, I saw the following error message:

Could not determine the dependencies of task ':onnxruntime-react-native:compileDebugAidl'.
> Could not resolve all task dependencies for configuration ':onnxruntime-react-native:debugCompileClasspath'.
   > Could not find com.microsoft.onnxruntime:onnxruntime-android:.
     Required by:
         project :onnxruntime-react-native

Possible solution:
 - Declare repository providing the artifact, see the documentation at https://docs.gradle.org/current/userguide/declaring_repositories.html

, @fs-eire do you by chance know what is the correct way to modify the React Native project configurations in order to use the full onnxruntime build? Any suggestions or reference will be greatly appreciated!

As a side note, we will look into providing a React Native package with a full build in the future.

This would be really fantistic, I really appreciate all you guys' hard work on incorporating the full onnxruntime build for both iOS and Android into the onnxruntime-react-native package.

This is specific to using ORT from C/C++, which I believe is not what you are trying to do. Let us know if that's not the case.

Aaah I see, thanks for the clarification! Yes you are correct, I am using onnxruntime-react-native instead of onnxruntime with C/C++, even though I see for the iOS backend, onnxruntime-react-native uses Objective-C++ to include the C/C++ headers with the line #import <onnxruntime/onnxruntime_cxx_api.h>, is that correct?

Consuming the iOS CocoaPods directly should work. I think this line would need to be updated to refer to 'onnxruntime-c' (the full build package) instead:

I see, thanks for the clarification!

edgchen1 commented 2 years ago

I see, thanks for the clarification! For my Android project, I tried changing the line implementation "com.microsoft.onnxruntime:onnxruntime-mobile:latest.integration@aar" to implementation "com.microsoft.onnxruntime:onnxruntime-android" in the MyProject/node_modules/onnxruntime-react-native/android/build.gradle file, but when I tried to run the program, I saw the following error message:

Could not determine the dependencies of task ':onnxruntime-react-native:compileDebugAidl'.
> Could not resolve all task dependencies for configuration ':onnxruntime-react-native:debugCompileClasspath'.
   > Could not find com.microsoft.onnxruntime:onnxruntime-android:.
     Required by:
         project :onnxruntime-react-native

Possible solution:
 - Declare repository providing the artifact, see the documentation at https://docs.gradle.org/current/userguide/declaring_repositories.html

Could you try replacing it with implementation "com.microsoft.onnxruntime:onnxruntime-android:latest.integration@aar" and see if that works?

Aaah I see, thanks for the clarification! Yes you are correct, I am using onnxruntime-react-native instead of onnxruntime with C/C++, even though I see for the iOS backend, onnxruntime-react-native uses Objective-C++ to include the C/C++ headers with the line #import <onnxruntime/onnxruntime_cxx_api.h>, is that correct?

That's correct. Though things are a bit different for iOS - the onnxruntime-c pod actually provides the ORT C/C++ API without the need to copy out header and library files.

jackylu0124 commented 1 year ago

Hi @edgchen1,

Apologies for the very late reply, but replacing it with implementation "com.microsoft.onnxruntime:onnxruntime-android:latest.integration@aar" works perfect! Thank you very much for the help and also the clarification!

edgchen1 commented 1 year ago

Glad to hear it works. I'll close this issue now.

BTW, starting from the next release (1.13), the React Native package will use a full ORT build instead of a reduced ops build. https://github.com/microsoft/onnxruntime/pull/13037

jackylu0124 commented 1 year ago

Hi @edgchen1,

Got it, thank you very much for the update and the work! I really appreciate it. On a different note, do you by chance know when the fix/change for issue #12500 will be integrated into the release (either dev or official)? Thank you very much again!

edgchen1 commented 1 year ago

Hi @edgchen1,

Got it, thank you very much for the update and the work! I really appreciate it. On a different note, do you by chance know when the fix/change for issue #12500 will be integrated into the release (either dev or official)? Thank you very much again!

No problem.

Unfortunately, as that's still in a PR (#12676) it probably won't make 1.13. I'd say keep an eye on the PR to see when it gets merged. Sorry, I don't have a better estimate.

jackylu0124 commented 1 year ago

Hi @edgchen1, Got it, thank you very much for the update and the work! I really appreciate it. On a different note, do you by chance know when the fix/change for issue #12500 will be integrated into the release (either dev or official)? Thank you very much again!

No problem.

Unfortunately, as that's still in a PR (#12676) it probably won't make 1.13. I'd say keep an eye on the PR to see when it gets merged. Sorry, I don't have a better estimate.

Got it, no problem, thanks a lot for all the help and information again!