[Mobile | iOS] I got "Unknown exception" error.

[edom18]

Describe the issue

I'm trying to use onnxruntime on Unity. I succeeded to use it on Unity and iOS device.

But I'm facing a problem. Firstly, I tried to use it on a new Unity project and built it to my iPhone then it worked. This was just a test to run onnxruntime on device.

I got a problem from here. I tried to port succeeded project (codes) to my actual project but it didn't work with "Unknown exception" error I attached.

I'm wondered why my actual project can't inference that model.

As you can see first log is a warning opset version and then it occurred "Unknown exception" error. I can't get any reasons from that. I really want to know where this error was happened.

I copied all of libraries and code that worked correctly on the test project. I'm guessing my actual project has some dependencies about CoreML or machine learning libraries because the project uses ARKit and ARDK.

Does anyone have any ideas to solve this problem? I'd like to know what even a little hint.

2023-09-28 14:59:05.079009+0900 APP_NAME[10908:7310568] 2023-09-28 14:59:05.078994 [W:onnxruntime:, model.cc:183 Model] ONNX Runtime only *guarantees* support for models stamped with opset version 7 or above for opset domain 'ai.onnx'. Please upgrade your model to opset 7 or higher. For now, this opset 1 model may run depending upon legacy support of some older opset version operators.
2023-09-28 14:59:05.080926+0900 APP_NAME[10908:7310568] 2023-09-28 14:59:05.080918 [E:onnxruntime:, inference_session.cc:744 LoadWithLoader] Unknown exception
OnnxRuntimeException: [ErrorCode:RuntimeException] Load model from /private/var/containers/Bundle/Application/C1A3814C-3DD9-48CC-929D-A7AC6852DF25/APP_NAME.app/Data/Raw/text-tokenizer/hoge.onnx failed:Encountered unknown exception in LoadWithLoader()
  at Microsoft.ML.OnnxRuntime.InferenceSession.Run (System.Collections.Generic.IReadOnlyCollection`1[T] inputNames, System.Collections.Generic.IReadOnlyCollection`1[T] inputValues, System.Collections.Generic.IReadOnlyCollection`1[T] outputNames, System.Collections.Generic.IReadOnlyCollection`1[T] outputValues, Microsoft.ML.OnnxRuntime.RunOptions options) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.OnnxRuntime.InferenceSession.Init (System.String modelPath, Microsoft.ML.OnnxRuntime.SessionOptions options, Microsoft.ML.OnnxRuntime.PrePackedWeightsContainer prepackedWeightsContainer) [0x00000] in <00000000000000000000000000000000>:0 
  at Tokenizer.TokenizeText (System.String text) [0x00000] in <00000000000000000000000000000000>:0 
  at ClipTest.Predict () [0x00000] in <00000000000000000000000000000000>:0 
  at UnityEngine.Events.UnityEvent.Invoke () [0x00000] in <00000000000000000000000000000000>:0 
  at UnityEngine.EventSystems.ExecuteEvents.Execute[T] (UnityEngine.GameObject target, UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1] functor) [0x00000] in <00000000000000000000000000000000>:0 
  at UnityEngine.EventSystems.StandaloneInputModule.ProcessTouchPress (UnityEngine.EventSystems.PointerEventData pointerEvent, System.Boolean pressed, System.Boolean released) [0x00000] in <00000000000000000000000000000000>:0 
  at UnityEngine.EventSystems.StandaloneInputModule.ProcessTouchEvents () [0x00000] in <00000000000000000000000000000000>:0 
  at UnityEngine.EventSystems.StandaloneInputModule.Process () [0x00000] in <00000000000000000000000000000000>:0 
UnityEngine.EventSystems.ExecuteEvents:Execute(GameObject, BaseEventData, EventFunction`1)
UnityEngine.EventSystems.StandaloneInputModule:ProcessTouchPress(PointerEventData, Boolean, Boolean)
UnityEngine.EventSystems.StandaloneInputModule:ProcessTouchEvents()
UnityEngine.EventSystems.StandaloneInputModule:Process()

To reproduce

I'm not sure because I can use onnxruntime correctly but I can't other project even if I set up the same way.

Urgency

No response

Platform

iOS

OS Version

iOS 16.6

ONNX Runtime Installation

Built from Source

Compiler Version (if 'Built from Source')

Xcode 15

Package Name (if 'Released Package')

None

ONNX Runtime Version or Commit ID

1.16.0

ONNX Runtime API

C#

Architecture

ARM64

Execution Provider

CoreML

Execution Provider Library Version

No response

[baijumeswani]

It is hard to tell what the error might be without a repro. But based on the logs you've attached:

2023-09-28 14:59:05.080926+0900 APP_NAME[10908:7310568] 2023-09-28 14:59:05.080918 [E:onnxruntime:, inference_session.cc:744 LoadWithLoader] Unknown exception
OnnxRuntimeException: [ErrorCode:RuntimeException] Load model from /private/var/containers/Bundle/Application/C1A3814C-3DD9-48CC-929D-A7AC6852DF25/APP_NAME.app/Data/Raw/text-tokenizer/hoge.onnx failed:Encountered unknown exception in LoadWithLoader()

The exception occurs at model load time. Is it possible that the model is not at the location specified? Are you able to share the snippet of the code that instantiates the Session object?

cc @edgchen1 to see if has encountered this error before.

[edom18]

Thank you for replying!

I checked the model location in Objective-C (Xcode). It's located the place that it was expected.

Here is a code of my implementation. I'm curious why it works correctly in other project. (This code is completely the same on other project that I tested.)

using System.Collections.Generic;
using System.Linq;
using UnityEngine;
using Microsoft.ML.OnnxRuntime;
using Microsoft.ML.OnnxRuntime.Tensors;

public static class Tokenizer
{
    public static int[] TokenizeText(string text)
    {
        // Create Tokenizer and tokenize the sentence.
        string tokenizerOnnxPath = $"{Application.streamingAssetsPath}/text-tokenizer/cliptok.onnx";

        // Create session options for custom op of extensions
        SessionOptions sessionOptions = new SessionOptions();
        sessionOptions.RegisterOrtExtensions();

        // // Create an InferenceSession from the onnx clip tokenizer.
        InferenceSession tokenizeSession = new InferenceSession(tokenizerOnnxPath, sessionOptions);

        DenseTensor<string> inputTensor = new DenseTensor<string>(new string[] { text }, new int[] { 1 });

        List<NamedOnnxValue> inputString = new List<NamedOnnxValue> { NamedOnnxValue.CreateFromTensor<string>("string_input", inputTensor) };

        // Run session and send the input data in to get inference output. 
        IDisposableReadOnlyCollection<DisposableNamedOnnxValue> tokens = tokenizeSession.Run(inputString);

        long[] inputIds = (tokens.ToList().First().Value as IEnumerable<long>).ToArray();

        Debug.Log(string.Join(" ", inputIds));

        // Cast inputIds to Int32
        int[] inputIdsInt = inputIds.Select(x => (int)x).ToArray();
        int modelMaxLength = 77;

        // Pad array with 49407 until length is modelMaxLength
        if (inputIdsInt.Length < modelMaxLength)
        {
            int[] pad = Enumerable.Repeat(49407, 77 - inputIdsInt.Length).ToArray();
            inputIdsInt = inputIdsInt.Concat(pad).ToArray();
        }

        return inputIdsInt;
    }
}

[edgchen1]

It's kind of odd to see this error message from the catch-all catch block. It appears that something got thrown that is not derived from std::exception.

https://github.com/microsoft/onnxruntime/blob/caf98128c1dbbee1ce3c831364cee88031471a32/onnxruntime/core/session/inference_session.cc#L743-L747

Not too much other info to go on from the logs there.

Could you try setting the OrtEnv log level to verbose and see if there's any more useful output?

[edom18]

@edgchen1 Thank you for replying! I didn't notice there is the log level. I'll try it!

[edom18]

I tried again with new log level but I got almost same logs.

OnnxRuntimeException: [ErrorCode:RuntimeException] Load model from /private/var/containers/Bundle/Application/F4FC2370-D6FB-4618-91AD-7B71BAD07023/APP_NAME.app/Data/Raw/text-tokenizer/cliptok.onnx failed:Encountered unknown exception in LoadWithLoader()
  at Microsoft.ML.OnnxRuntime.InferenceSession.Run (System.Collections.Generic.IReadOnlyCollection`1[T] inputNames, System.Collections.Generic.IReadOnlyCollection`1[T] inputValues, System.Collections.Generic.IReadOnlyCollection`1[T] outputNames, System.Collections.Generic.IReadOnlyCollection`1[T] outputValues, Microsoft.ML.OnnxRuntime.RunOptions options) [0x00000] in <00000000000000000000000000000000>:0 
  at Microsoft.ML.OnnxRuntime.InferenceSession.Init (System.String modelPath, Microsoft.ML.OnnxRuntime.SessionOptions options, Microsoft.ML.OnnxRuntime.PrePackedWeightsContainer prepackedWeightsContainer) [0x00000] in <00000000000000000000000000000000>:0 
  at Tokenizer.TokenizeText (System.String text) [0x00000] in <00000000000000000000000000000000>:0 
2023-09-30 15:57:58.225971 [I:onnxruntime:, inference_session.cc:328 operator()] Flush-to-zero and denormal-as-zero are off
2023-09-30 15:57:58.226295 [I:onnxruntime:, inference_session.cc:336 ConstructorCommon] Creating and using per session threadpools since use_per_session_threads_ is true
2023-09-30 15:57:58.226312 [I:onnxruntime:, inference_session.cc:354 ConstructorCommon] Dynamic block base set to 0
2023-09-30 15:57:58.232618 [W:onnxruntime:, model.cc:183 Model] ONNX Runtime only *guarantees* support for models stamped with opset version 7 or above for opset domain 'ai.onnx'. Please upgrade your model to opset 7 or higher. For now, this opset 1 model may run depending upon legacy support of some older opset version operators.
2023-09-30 15:57:58.234723 [E:onnxruntime:, inference_session.cc:744 LoadWithLoader] Unknown exception
  at ClipTest.Predict () [0x00000] in <00000000000000000000000000000000>:0 
  at UnityEngine.Events.UnityEvent.Invoke () [0x00000] in <00000000000000000000000000000000>:0 
  at UnityEngine.EventSystems.ExecuteEvents.Execute[T] (UnityEngine.GameObject target, UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1] functor) [0x00000] in <00000000000000000000000000000000>:0 
  at UnityEngine.EventSystems.StandaloneInputModule.ProcessTouchPress (UnityEngine.EventSystems.PointerEventData pointerEvent, System.Boolean pressed, System.Boolean released) [0x00000] in <00000000000000000000000000000000>:0 
  at UnityEngine.EventSystems.StandaloneInputModule.ProcessTouchEvents () [0x00000] in <00000000000000000000000000000000>:0 
  at UnityEngine.EventSystems.StandaloneInputModule.Process () [0x00000] in <00000000000000000000000000000000>:0 
UnityEngine.EventSystems.ExecuteEvents:Execute(GameObject, BaseEventData, EventFunction`1)
UnityEngine.EventSystems.StandaloneInputModule:ProcessTouchPress(PointerEventData, Boolean, Boolean)
UnityEngine.EventSystems.StandaloneInputModule:ProcessTouchEvents()
UnityEngine.EventSystems.StandaloneInputModule:Process()

[skottmckay]

2023-09-30 15:57:58.232618 [W:onnxruntime:, model.cc:183 Model] ONNX Runtime only guarantees support for models stamped with opset version 7 or above for opset domain 'ai.onnx'. Please upgrade your model to opset 7 or higher. For now, this opset 1 model may run depending upon legacy support of some older opset version operators.

I'd deal with this warning first. Opset 1 is a very strange opset to see in a model given that is from many years ago so I'd suspect the model could be invalid. How was the model created? Are you able to share it?

[edom18]

Thank you for replying! I'm curious why this error was occurred because I used ai.onnx v11. (Please see an image that I attached.)

AND, I'm also curious I could use this model other project that I tested and this project didn't occur this warning (also it worked fine. This is sooo curious.)

[skottmckay]

Not sure how your model was created but CLIPTokenizer is not an ONNX operator.

There is a CLIPTokenizer custom operator in the onnxruntime-extensions library. Are you referencing that package as well? If it's C# that's this nuget package: https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime.Extensions

Based on the error message and the model imports I'm guessing the intention is to use the custom operator but the domain for the node is not set to 'ai.onnx.contrib' so it thinks CLIPTokenizer is an ONNX operator in opset 1. I'd still expect a better error message from the exception though.

You said onnxruntime was built from source but for C# that would involve building a nuget package with runtime binaries for different platforms like Android and iOS which is a complicated process. Is that what you did, or are you using the Microsoft.ML.OnnxRuntime nuget package and just building a C# app from source?

[edom18]

Thank you for replying!

Firstly, I have completed running the models in other project. This is the curious point why I can't do that in another project.

I'll answer your questions:

There is a CLIPTokenizer custom operator in the onnxruntime-extensions library. Are you referencing that package as well?

Yes, I know the tokenizer needs to use extension operators so I imported extension binaries into my project.

Is that what you did, or are you using the Microsoft.ML.OnnxRuntime nuget package and just building a C# app from source?

I imported OnnxRuntime binaries from the NuGet package. I also imported C# wrapper files from its GitHub repository into my project. Finally, I built OnnxRntime extensions for macOS and iOS and imported them to my project.

These ways lead me to work correctly in my test project and after that I ported them to my other actual project that I want to use them but it didn't work correctly with the errors I attached.

I think these binaries and files are completely the same because I copied them from test project to actual project.

[skottmckay]

I also imported C# wrapper files from its GitHub repository into my project. ... I built OnnxRntime extensions for macOS and iOS and imported them to my project.

Not sure what you mean by this. Can you please clarify?

Typically a project would reference the following nuget packages and there would be no 'wrapper files' to import from GitHub repos or custom builds required.

• Microsoft.ML.OnnxRuntime (native C++ libraries),
• Microsoft.ML.OnnxRuntime.Managed (C# bindings),
• Microsoft.ML.OnnxRuntime.Extensions (custom ops) if required

Could you also please share the model?

[edom18]

I referenced this article (sorry this is written in Japanese.) https://www.hanachiru-blog.com/entry/2022/06/30/120000

This article said he had to import Microsoft.ML.OnnxRuntime repository source codes and also had to import its runtime binary for iOS platform dependency. This is because he had to modify a few code to adapt Unity C#.

It was my mistake it was not 'wrapper files'. I needed to modify a few source codes to use its binary.

Here is a tokenizer model that I'm using. https://drive.google.com/file/d/1H643ex4Vr125Z2HVh56bNcD3ixP1hF9r/view?usp=sharing

Here is files that I imported to my Unity project.

Microsoft.ML.OnnxRuntime is source codes from the repository. Others are binaries.

[skottmckay]

I loaded the model and stepped through the opset validation and I didn't get a warning about the model attempting to use opset 1 for onnx. It correctly found the domain of 'ai.onnx.contrib' for the CLIPTokenizer operator. That warning is only possible if the domain is empty.

https://github.com/microsoft/onnxruntime/blob/e11849e716d7d63d0ff2eee8e66b93406c1ef547/onnxruntime/core/graph/model.cc#L181-L190

How does this model differ from the one you got the warning with?

[edom18]

Thank you for testing!

How does this model differ from the one you got the warning with?

Unfortunately, it is completely the same.

I'm going to test the model from creating new project and adding frameworks step by step.