Open kipcole9 opened 3 months ago
Thanks for the idea! We've been planning to implement model loading from memory for a while now, but we're waiting on the upstream ort
2.0 changes to settle before doing so.
One other benefit that will unlock is loading directly from axon_onnx, so in the future we may be able to export Axon models to run with Ortex in memory.
Thanks for the consideration - and the library. Feel free to close since its already on your to-do list.
We'll keep this open to track, should be an easy add once the ort
API settles
In working on a segment-anthing livebook, based upon this forum example, I am downloading the onnx model from hugging face. The benefit is that the Livebook can be readily shared with no dependencies.
Currently it needs to be saved to a file system before calling
Ortext.load/1
since that function ultimately delegates to the following in model.rs:A version or an option which calls
.with_model_from_memory(binary)
would be a valuable, and hopefully a simple addition.I think Livebooks is a likely a common use case so hopefully something you might consider. I would happily send a PR if I had any relevant skill, but in Rust I do not.