search

Anush008 / fastembed-go

Go implementation of @qdrant/fastembed.
https://pkg.go.dev/github.com/anush008/fastembed-go
MIT License
45 stars 4 forks source link

Using GPU #9

Closed msj121 closed 2 months ago

msj121 commented 7 months ago

Looking through the code I didn't see a setting for using Cuda/GPU, is this done automatically, or is there a setting? Tried looking in the Onnx project, but got a little confused.

Thanks.

Anush008 commented 7 months ago

Hi. We could add a configuration option for GPU, like FastEmbed for the other languages. But since I don't have access to a GPU instance, I can't verify it. If you do, please consider contributing.

msj121 commented 7 months ago

Ok - I do have indirect access to a computer with a gpu. Let me try and look around the code etc... If you have any advice lmk.

Anush008 commented 7 months ago

There's an execution providers option. https://github.com/Anush008/fastembed-go/blob/bbcc4da928d23eea76be309148ad1d298b9c23af/fastembed.go#L61 that is not being used.

You'd need to verify if we can pass and make it work with CUDA for example. We're using https://github.com/yalue/onnxruntime_go to interface with ONNX.

Anush008 commented 7 months ago

Probably would have to change the type of the option from string[] to the one used by onnxruntime_go for execution providers.

Anush008 commented 2 months ago

Closing since stale. Please feel free to re-open.