Using ML.Net and ONNX in Alpine Docker gives library load error.

[Webreaper]

Describe the bug I'm using ML.Net and ONNX to process object detection with a Yolo onnx model. It works fine on my M1 Macbook pro. But when I deploy it to linux, I get this error:

The type initializer for 'Microsoft.ML.OnnxRuntime.NativeMethods' threw an exception.
 ---> System.DllNotFoundException: Unable to load shared library 'onnxruntime' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: Error loading shared library libonnxruntime: No such file or directory
   at Microsoft.ML.OnnxRuntime.NativeMethods.OrtGetApiBase()
   at Microsoft.ML.OnnxRuntime.NativeMethods..cctor() in Microsoft.ML.OnnxRuntime.dll:token 0x600006e+0x1
   --- End of inner exception stack trace ---
   at Microsoft.ML.OnnxRuntime.SessionOptions..ctor() in Microsoft.ML.OnnxRuntime.dll:token 0x60000f3+0x6f
   at Microsoft.ML.OnnxRuntime.InferenceSession..ctor(String modelPath) in Microsoft.ML.OnnxRuntime.dll:token 0x6000021+0x2c
   at Yolov5Net.Scorer.YoloScorer`1..ctor() in Damselfly.ML.ObjectDetection.dll:token 0x6000020+0x34
   at Damselfly.ML.ObjectDetection.ObjectDetector..ctor() in Damselfly.ML.ObjectDetection.dll:token 0x6000001+0x15

I've updated the Dockerfile to install libgomp as I've seen recommended around the place (e.g., here) but it doesn't seem to work. I'm not sure if this is something Alpine-related.

My Dockerfile is here.

Urgency Hard deadlines: none

System information

• Linux version: Synology linux v3.10.105, running Docker
• ONNX Runtime installed from (source or binary): binary
• ONNX Runtime version: Not sure
• Python version: N/A
• CLR Version: 6.0 preview 5
• ML.Net version: 1.8.0
• Visual Studio version (if applicable): VS for Mac v8.10
• GCC/Compiler version (if compiling from source): N/A
• CUDA/cuDNN version: N/A
• GPU model and memory: N/A
• ONNX Model: Yolov5

Expected behavior No library exception, and the Inference initialises correctly.

[yuslepukhin]

Cc: @RyanUnderhill

[Webreaper]

If it helps, this is basically the project I'm trying to dockerise in Alpine, as part of a large app. https://github.com/mentalstack/yolov5-net

Small example repro project here: https://github.com/Webreaper/MLONNXDockerRepro

[snnn]

I'm waiting manylinux's musl support. Once it's done, I will resolve this.

[Webreaper]

Thanks. No urgency from me, as I've switched to an Ubuntu image (to solve this issue and because I also use EmguCV which also doesn't work on alpine).

[snnn]

To track: https://github.com/pypa/manylinux/issues/1134 https://github.com/pypa/auditwheel/issues/305

[stale[bot]]

This issue has been automatically marked as stale due to inactivity and will be closed in 7 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

[snnn]

We don't support alpine yet. We can leverage manylinux docker images to provide such a python package first. If the package works good, we may start to work on adding our own docker files for alpine and build such things for .Net/Java/... .

@faxu, I will assign the item to you now.

[mfrancisc]

Hello, any update on this? Any available workaround on how to make it work on Alpine? Thanks.

[sunjingsong]

Hello, any update on this? I met same problem on Alpine Linux to deploy ML on java server

[ChristophHornung]

I ran into the same issue when trying to dockerize my code into an alpine image.

The way I solved it was to first install the onnxruntime package

RUN apk add --no-cache onnxruntime --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing/ -X http://dl-cdn.alpinelinux.org/alpine/edge/community/ -X http://dl-cdn.alpinelinux.org/alpine/edge/main

for some reason this would still give the same "Unable to load library" error. I also had to copy the onnxruntime.so into the same folder as my app

RUN cp /usr/lib/libonnxruntime.so.1.15.1 /app/onnxruntime.so

[jfox454]

In my case I was trying to run ML.NET and ONNX in an Azure Function deployed on an Linux instance of Azure App Service.

I was getting the Exception

System.DllNotFoundException: Unable to load shared library 'onnxruntime' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libonnxruntime: cannot open shared object file: No such file or directory

When attempting to apply an ONNX model.

The solution was to copy the libonnxruntime.so file from the deployment /runtimes/linux-x64/native/ folder to the deployment /bin/ folder.