Unable to load shared library 'onnxruntime' on MacOS (DllNotFoundException)

[leXorNet]

Crash when trying to load onnxruntime when running console application on osx with .net6 core and macos workload. Dll's looks to be in the bin folder, but can't be found/loaded. No macos versions available in the nuget package?

Reproduction steps: Create a new .netcore 6 console application. Add macOS workload and set target framework to 'net6.0-macos'. Add Microsoft.ML and OnnxRuntime from nuget. Add some code to call .ApplyOnnxModel(...). -> Crash

Setup: macOS 11.6 (x64) Visual Studio Mac 2022 (preview 1->3) OnnxRuntime 1.9.0 (from nuget) .NetCore 6.0 with macOS workload

Callstack:

Unhandled exception. System.TypeInitializationException: The type initializer for 'Microsoft.ML.OnnxRuntime.NativeMethods' threw an exception.
 ---> System.DllNotFoundException: Unable to load shared library 'onnxruntime' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libonnxruntime, 1): image not found
   at Microsoft.ML.OnnxRuntime.NativeMethods.OrtGetApiBase()
   at Microsoft.ML.OnnxRuntime.NativeMethods..cctor()
   --- End of inner exception stack trace ---
   at Microsoft.ML.OnnxRuntime.SessionOptions..ctor()
   at Microsoft.ML.Transforms.Onnx.OnnxModel..ctor(String modelFile, Nullable`1 gpuDeviceId, Boolean fallbackToCpu, Boolean ownModelFile, IDictionary`2 shapeDictionary, Int32 recursionLimit, Nullable`1 interOpNumThreads, Nullable`1 intraOpNumThreads)
   at Microsoft.ML.Transforms.Onnx.OnnxTransformer..ctor(IHostEnvironment env, Options options, Byte[] modelBytes)
   at Microsoft.ML.Transforms.Onnx.OnnxTransformer..ctor(IHostEnvironment env, String[] outputColumnNames, String[] inputColumnNames, String modelFile, Nullable`1 gpuDeviceId, Boolean fallbackToCpu, IDictionary`2 shapeDictionary, Int32 recursionLimit, Nullable`1 interOpNumThreads, Nullable`1 intraOpNumThreads)
   at Microsoft.ML.Transforms.Onnx.OnnxScoringEstimator..ctor(IHostEnvironment env, String[] outputColumnNames, String[] inputColumnNames, String modelFile, Nullable`1 gpuDeviceId, Boolean fallbackToCpu, IDictionary`2 shapeDictionary, Int32 recursionLimit, Nullable`1 interOpNumThreads, Nullable`1 intraOpNumThreads)
   at Microsoft.ML.OnnxCatalog.ApplyOnnxModel(TransformsCatalog catalog, String[] outputColumnNames, String[] inputColumnNames, String modelFile, Nullable`1 gpuDeviceId, Boolean fallbackToCpu)

[yuslepukhin]

Use otool to find out what are the dependencies that onnxruntime requires on MacOs and then recursively see if they miss any of their dependancies.

[atkinsonbg]

@leXorNet Try adding the Microsoft.ML.OnnxRuntime package, this solved it for me on Macos.

https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime/

[niyazidageek]

@leXorNet the problem is that it doesn't work on mac OS in particular.

[juwens]

I can confirm the error with a M1 Mac. Does the cpu arch matter here? @atkinsonbg and @niyazidageek and @lexornet : what's your cpu arch?

.NET SDK (reflecting any global.json):
 Version:   6.0.100
 Commit:    9e8b04bbff

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  12.0
 OS Platform: Darwin
 RID:         osx-arm64
 Base Path:   /usr/local/share/dotnet/sdk/6.0.100/

Host (useful for support):
  Version: 6.0.0
  Commit:  4822e3c3aa

.NET SDKs installed:
  6.0.100 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

On the website it looks like arm is not (yet) supported for mac.

[juwens]

Or it might be caused by the fact, that the native libonnxruntime.dylib is bundled in the nuget for only for osx 10.14

[niyazidageek]

@juwens I solved the problem. What i did was installing "Microsoft.ML.Mkl.Redist" package from nuget.org. I am not sure what the problem actually is about, but it finally works. What worked for me more: installing "Microsoft.ML.OnnxRuntime.MKLML" package(int this case without "Microsoft.ML.Mkl.Redist"). Issue still is mysterious. I think i would better not touch any packages regarding onnx so that i won't have any problems in the future. Thank you!

[niyazidageek]

@leXorNet I think the issue can be closed.

[juwens]

@leXorNet thanks for the workaround.

The packaging of the nuget needs to be fixed for osx anyways.

the lib must be placed in runtimes/osx-x64/native/, but is currently placed in runtimes/osx.10.14-x64/native

respective for arm: the lib must be placed in runtimes/osx-arm64/native/, but is currently placed in runtimes/osx.10.14-arm64/native

If somebody is interested:

• MKL == Math Kernel Library
• installing Microsoft.ML.Mkl.Redist package does not fix the issue, because it only contains a "libMklProxyNative.dylib" for osx; see nuget.info: Microsoft.ML.Mkl.Redist
• installing Microsoft.ML.OnnxRuntime.MKLML is only a workaround for intel macs (or x64 .net runtime on arm mac), because it contains the necessary native lib libonnxruntime.dylib, but only for x64 not arm64, and it contains unnecessary blis nuget.info: Microsoft.ML.OnnxRuntime.MKLML

[niyazidageek]

@juwens Exactly!

[juwens]

@niyazidageek which cpu arch do you have? Intel x64 or Apple Silicon aka arm64?

Please post the output of dotnet --info, it contains this information. That would be pretty helpful, to narrow down the problem.

[niyazidageek]

@juwens Mine is Apple Silicon aka arm64. Here is the information:

[juwens]

@juwens Mine is Apple Silicon aka arm64. Here is the information:

thanks for the info. As i thought, you are using a x86 runtime as you can see in the output (because Net 5 has no arm runtime) despite having a ARM Mac.

Install the Net 6 arm64 Runtime and you will see the problem again, even with the additional Microsoft.ML.OnnxRuntime.MKLML.

[khalidabuhakmeh]

I was able to work around this issue on macOS by using homebrew to install the onnxruntime dependency.

brew install onnxruntime

It lets me run workloads, but I would expect the package to bring down the latest .dll.

[niyazidageek]

@khalidabuhakmeh Wow! Thank you for the solution!!! And also, installing previous version of onnxruntime helped me as well. I had 1.10.0, which was not working on Mac OS, however, 1.8.1 runs perfectly. Anyways, thank you!

[khalidabuhakmeh]

@niyazidageek I still think something is wrong with the NuGet package because the .dylib is in the bin directory but just not linking correctly. Luckily installing the onnxruntime via homebrew links it correctly machine-wide.

It's enough to get some code running at least. 😅

[niyazidageek]

@leXorNet I think you can close the issue. Have those methods helped you?

[khalidabuhakmeh]

I wouldn't close this issue, the problem still exists in the NuGet package. There doesn't seem to be a build target for macOS environments, which is what is failing to copy the libonnxruntime.dylib from the runtimes folder into the root of the build directory.

The issue is a missing build target for macOS outside of Xamarin targets.

---

Update: Looking at the targets under the csharp directory, I noticed there is no mention of either macOS or Linux platforms under the netstandard directory.

That means this library is effectively broken for folks looking to use it in those environments.

The targets need updating to support more platforms, as right now it is hardcoded with a Windows environment in mind.

https://github.com/microsoft/onnxruntime/blob/master/csharp/src/Microsoft.ML.OnnxRuntime/targets/netstandard/props.xml

I'm not exactly sure how or what to do to fix it, but identifying it is half the problem, right? 😅

[baronfel]

I believe the culprit is this line in generate_nspec_for_native_nuget.py. This code is responsible for creating a nuspec that is then packaged, and it creates the native runtimes folders for MacOS in a way that is specific to 10.14, as recorded above by @juwens. If this were changed to just osx-%s, you'd get a package that works correctly for all OSX consumers (at least barring any actual OS requirements on the part of the native code). This would be in line with how the win and linux platforms are provided as general baselines as well.

To test this, users can manually copy the osx.10.14-x64 directory from the package runtimes in their nuget cache (~/.nuget/package/microsoft.ml.onnxruntime/1.10.0 is the default location) to a sibling folder called osx-x64, then purge their obj and bin directories and re-publish their applications. At this point the application should run. I verified this offline with @khalidabuhakmeh.

[juwens]

@baronfel thanks, I was also searching for this line, but without success. Thanks for the PR 👌

[baronfel]

Brief update - my earlier comment is in fact actually wrong! The osx.10.14 constraint in the NuGet package establishes a lower bound of support, and was added for a reason. There's something we need to do in the SDK/Build/Runtime side of things to make make this scenario work. I'll keep running this down, and when we have a root cause or plan I'll be sure to link it here. In the mean time, for users of this package, if you're on an OSX version greater than 10.14, continue to explicitly specify your runtime identifier via either setting the RuntimeIdentifier MSBuild property in your project files, or explicitly on your command lines via --os osx.<version number>.

[baronfel]

If you're having this issue today, can you run your application with the COREHOST_TRACE=1 and COREHOST_TRACEFILE=host.txt environment variables? This will produce a log file from the runtime itself that documents the probing paths that it's trying to resolve native references from, and may help narrow down the problem space.

[stale[bot]]

This issue has been automatically marked as stale due to inactivity and will be closed in 7 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[silveredgold]

I'm running into this issue as well and while I get someone to get a core host trace file, the error messages include at least some of the paths being checked:

System.DllNotFoundException: Unable to load shared library 'onnxruntime' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libonnxruntime, 0x0001): tried: 'libonnxruntime' (no such file), '/usr/local/lib/libonnxruntime' (no such file), '/usr/lib/libonnxruntime' (no such file), '/Users/username/Documents/username2022/Coding/Beta/appname-osx-x64/libonnxruntime' (no such file)
         at Microsoft.ML.OnnxRuntime.NativeMethods.OrtGetApiBase()
         at Microsoft.ML.OnnxRuntime.NativeMethods..cctor()
         --- End of inner exception stack trace ---
         at Microsoft.ML.OnnxRuntime.SessionOptions..ctor()

[faxu]

@silveredgold what ORT version are you using?

[skottmckay]

.net 6 support was added in the most recent release - 1.12 - and net6.0-macos is included as a target framework.

@silveredgold maybe create a new issue with full details of your setup such as ORT version, which nuget packages you're using, what framework/s you're targeting etc.

[silveredgold]

I was using 1.12 with net6.0 (publishing for osx-x64 and osx-arm64) but I'll create a separate issue.

[ronimizy]

brew install onnxruntime and setting environment variable like so worked for me (macOS 13.0, arm64):

DYLD_LIBRARY_PATH=/opt/homebrew/lib

Note that it works only if you will pass it to executable directly, I had to add it to my Rider's executable configuration, adding it to OS environment variable, using .zshrc for example, wouldn't do anything. In fact Environment.GetEnviromentVariable("DYLD_LIBRARY_PATH") returned null when doing so.

It seems to me, as it is a dotnet runtime issue, as exception says that it tried searching specific paths, and they are probably somewhat hardcoded for macOS, and PATH environment variable not being respected. For Apple Silicon Macs, homebrew does not install references into /usr/local/bin anymore, so it could be a source of this failure.

[armannaj]

I'm having the same issue using macOS Ventura 13.0.1 with Intel cpu. I tried all the above solutions but none of them worked.

I'm using Microsoft.ML.OnnxRuntime 1.13.1 in .Net 6 Console application.

Edit: I downgraded this package to 1.12.1 and it works fine for me.

[TrianecT-Wouter]

I also encountered this problem and solved it with a workaround. Seems to be an issue with the local NuGet / project(?)

System.DllNotFoundException occurs often after I update my code from GitHub using pull, build and then run the application in the emulator. The build then completes without errors, however I do get a warning that some onnxruntime dll already exists in my application apk file. When running the application in the emulator: no problem, until the ONNX models are loaded; this then crashes the app, because ONNX runtime is not available.

What did not work for me:

• reinstalling the package with dotnet add package Microsoft.ML.OnnxRuntime
• dotnet restore

What did work for me (multiple times; very reproducible):

• install a random package (e.g. dotnet add package System.Text.Json)
• uninstall that package again dotnet remove package System.Text.Json (not required)

My system: Apple M3 Pro, running Sanoma 14.2.1 Visual Studio Code for Mac 17.6.9 (build 415) Project: .NET MAUI 8.0.7 project; net8.0-android target and net8.0 target