Giskard-AI / giskard

🐒 Open-Source Evaluation & Testing for ML & LLM systems
https://docs.giskard.ai
Apache License 2.0
4.07k stars 267 forks source link

Set any groq model as default LLMClient for giskard #1977

Open Vikas9758 opened 4 months ago

Vikas9758 commented 4 months ago

πŸš€ Feature Request

Add Support for Groq in Giskard or show it in the documentation if already supported

πŸ”ˆ Motivation

The integration of Groq as an alternative to OpenAI in the Giskard library is highly beneficial. Groq offers speed that is almost equivalent or even faster than GPT-4 and significantly faster than any Ollama model. This will provide users with a high-performance alternative for their machine learning tasks.

πŸ›° Alternatives

Currently, the only alternative is using OpenAI's models, which may not meet all users' needs for speed and performance. Can use Ollama as well but using it is a lot of hassle especially in colab or kaggle notebooks.Incorporating Groq provides a competitive edge and additional flexibility.

πŸ“Ž Additional context

Here is the proposed code to integrate Groq with Giskard:

from groq import `Groq`
from giskard.llm.client.openai import OpenAIClient
import giskard as gsk

_client = Groq(api_key="YOUR_GROQ_API_KEY")
oc = OpenAIClient(model="any_model_in_groq", client=_client)
gsk.llm.set_default_client(oc)

This simple addition allows users to set up Groq as their default client in Giskard, enabling them to take advantage of Groq's superior speed and performance. Detailed documentation and examples will also be necessary to guide users through the setup and usage process.

henchaves commented 1 month ago

Hey @Vikas9758, thanks for your suggestion! Indeed it would be very useful to use Groq API itself instead of implementing a custom client. Would you be interested in implementing this?