search

elixir-nx / ortex

ONNX Runtime bindings for Elixir
MIT License
134 stars 16 forks source link

upgrading to ort 2.0.0-rc.2, onnxruntime 1.17.3 #28

Closed michaelbearne closed 1 week ago

michaelbearne commented 7 months ago

Hi

I had an issue getting the Ortex compiled in a Docker container, and then an issue running the tests on a Mac, which I thought might have been caused by downloading the Onnx runtime shared libraries.

So, I tried upgrading to the latest Rustler and Ortex. With these updates, I don't seem to need the libs copy workaround in lib/vortex/native.ex as now one lib is created. I have only tested this on Mac and Linux without Cuda.

The issue with not working on Mac was the Statically linking to CoreML bindings needing the link flag
"-C link-arg=-fapple-link-rtlib" according to the Ort docs.

The docker compile was a path issue with cargo, which I only figured out after updating the dependencies.

gregszumel commented 6 months ago

Hey @michaelbearne -- thanks for this PR!

I tried running mix test on my machine and had some lib-not-found issues, so maybe ort is changing the way it handles downloading libraries between rc-0 and rc-2?

I think we'll unfortunately need to hold off on merging this into main for now, at least until ort's API is more settled, or until I get more capacity to debug why I'm getting those errors.

nickkaltner commented 4 weeks ago

Is this likely to be merged anytime soon? I was looking into hugging face's model, smol v2 - https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct It has onnx versions, but they need IR v 10 image

I'm a bit of a newbie in this area but i like the idea of using onnx models on my mac :)

mortont commented 1 week ago

I'm going to close this since the fix is now in main, feel free to open another PR or issue if your issue persists!