Integrate GoMLX as another backend to run pipelines ?

[janpfeifer]

hi all,

I just bumped into hugot project by coincidence today, when looking for a library to load HF models in Go.

I maintain GoMLX - github.com/gomlx/gomlx a machine learning framework in Go powered by OpenXLA same used by Jax/Tensorflow, and in some cases even pytorch, also kind of an alternative to Onnx. But GoMLX also has the concept of backend, and ONNX is one of my next backed candidates ...

I thought I would say hi and maybe start a discussion of ways maybe the projects could collaborate ?

GoMLX does have auto differentiation, so it may make the idea of fine-tuning easier in Go as well. Well, the idea one could build the full model in Go if one wants, and ditch Python.

A couple of other related projects worth mentioning:

• GoMLX Gemma: I just got it running, as a first LLM/GenAI example for GoMLX.
• gonb: Jupyter Notebook for Golang -- works with CGO, may come in handy for data scientists using Go.
• gopjrt: Barebones wrapper for XLA (JIT-compiler), this is what GoMLX uses, and could be used directly as well. I use it for CPU, GPU (NVidia) and likely it will work for anyone who have binary distributions of OpenXLA PJRT plugins (there is one for Google's TPUs and Intel has one as well). All accessible as well from GoMLX.

cheers

[RJKeevil]

Hi @janpfeifer, sounds very interesting! lots to discuss about what this could look like, but always happy to support anything that brings the go/ML landscape forward. How is best to collaborate, should be have a call to discuss?

[janpfeifer]

hi @RJKeevil , the idea of a quick call would be great. I'll follow up by email, I hope it's ok.

[riccardopinosio]

Hi @janpfeifer, indeed I think the first easier step would probably be to make hugot and goMLX compatible so that models trained in goMLX can be run for inference in hugot pipelines seamlessly. This could be done by either exporting goMLX models to onnx (in which case hugot can just load those) or supporting goMLX models more directly. Then hugot could appear as a pipeline layer on top of goMLX. One pointer here is that hugot relies on/wraps around https://github.com/yalue/onnxruntime_go for onnx inference.

[gregfurman]

Perhaps related https://github.com/knights-analytics/hugot/issues/59. Onnx inference can be done using onnx-go (actively maintained as of Septemeber 2024) without an external onnxruntime.