Feat/groq

[bricesanchez]

Hi @andreibondarev,

Here you could find a PoC of using Groq Cloud: https://console.groq.com/login for inference.

Also because Groq does not support (yet?) embbedding. i give it the ability to pass a Hugging Face client to the embed in order to have a full usage of Groq with a RAG.

If the solution looks promising for you, i could spend time to write specs and in order to make it merged.

[andreibondarev]

Also because Groq does not support (yet?) embbedding. i give it the ability to pass a Hugging Face client to the embed in order to have a full usage of Groq with a RAG.

@bricesanchez I find this to be a bit odd but I understand what you're trying to solve. I'm wondering if there's a better approach? This one feels a bit hacky to me.

Also -- how much effort would it be to not use the groq gem and make the HTTP API calls directly with Net::HTTP? I'm only asking because we have so many dependencies already.

[bricesanchez]

@andreibondarev I took inspiration on what they did for this example: https://github.com/definitive-io/presidential-speeches-rag/blob/main/app.py#L89

At this time and because i'm currently learning LLM/RAG, it was the only way i found to do this based on your current implementation.

Also -- how much effort would it be to not use the groq gem and make the HTTP API calls directly with Net::HTTP? I'm only asking because we have so many dependencies already.

I understand your concern about adding dependencies.

At least in this case, it only check for require if Groq class is initialized, it does not add dependencies in the gempsec. It follows the pattern of your other LLM classes and for me it makes sense to do not re-implement the client of Groq and let an external library to do this.

Any thoughts?

[andreibondarev]

At this time and because i'm currently learning LLM/RAG, it was the only way i found to do this based on your current implementation.

I think one way to do this is to monkey patch the class with def embed() in your own code. Not a perfect solution either but I think it's better than modifying the library itself.

At least in this case, it only check for require if Groq class is initialized, it does not add dependencies in the gempsec. It follows the pattern of your other LLM classes and for me it makes sense to do not re-implement the client of Groq and let an external library to do this.

Let's do it 🚀