knights-analytics / hugot

Huggingface transformer pipelines in Golang
Apache License 2.0
283 stars 13 forks source link

Integrate GoMLX as another backend to run pipelines ? #50

Open janpfeifer opened 2 months ago

janpfeifer commented 2 months ago

hi all,

I just bumped into hugot project by coincidence today, when looking for a library to load HF models in Go.

I maintain GoMLX - github.com/gomlx/gomlx a machine learning framework in Go powered by OpenXLA same used by Jax/Tensorflow, and in some cases even pytorch, also kind of an alternative to Onnx. But GoMLX also has the concept of backend, and ONNX is one of my next backed candidates ...

I thought I would say hi and maybe start a discussion of ways maybe the projects could collaborate ?

GoMLX does have auto differentiation, so it may make the idea of fine-tuning easier in Go as well. Well, the idea one could build the full model in Go if one wants, and ditch Python.

A couple of other related projects worth mentioning:

cheers

RJKeevil commented 2 months ago

Hi @janpfeifer, sounds very interesting! lots to discuss about what this could look like, but always happy to support anything that brings the go/ML landscape forward. How is best to collaborate, should be have a call to discuss?

janpfeifer commented 2 months ago

hi @RJKeevil , the idea of a quick call would be great. I'll follow up by email, I hope it's ok.

riccardopinosio commented 2 months ago

Hi @janpfeifer, indeed I think the first easier step would probably be to make hugot and goMLX compatible so that models trained in goMLX can be run for inference in hugot pipelines seamlessly. This could be done by either exporting goMLX models to onnx (in which case hugot can just load those) or supporting goMLX models more directly. Then hugot could appear as a pipeline layer on top of goMLX. One pointer here is that hugot relies on/wraps around https://github.com/yalue/onnxruntime_go for onnx inference.