Groq API does not work.

[mmuyakwa]

No idea where it gets that URL-path, but it's not correct. I can't find it in the code.

INFO - HTTP Request: POST https://api.groq.com/openai/v1/openai/v1/chat/completions "HTTP/1.1 404 Not Found"
ERROR - Failed to generate response: Error code: 404 - {'error': {'message': 'Unknown request URL: POST /openai/v1/openai/v1/chat/completions. Please check the URL for typos, or see the docs at https://console.groq.com/docs/', 'type': 'invalid_request_error', 'code': 'unknown_url'}}
INFO - Response: Error in generating response

1. The Groq-Module is being used, so it does not need to use the alternative URL for OpenA-Compatibility.
2. And even if, the URL should be https://api.groq.com/openai/v1.

Documentation: https://console.groq.com/docs/openai

Something is fishy here.

[mmuyakwa]

My workaround for now is using the OpenAI-method and changing the base_urlto the compatible URL from GROQ.

Added "base_url" in voice_assistant/response_generation.py.

    try:
        if model == 'openai':
            client = OpenAI(api_key=api_key, base_url="https://api.groq.com/openai/v1") # !Added the base_url here!
            response = client.chat.completions.create(
                model=Config.OPENAI_LLM,
                messages=chat_history

Made significant changes in voice_assistant/config.py.

    # Model selection
    TRANSCRIPTION_MODEL = 'deepgram'  # possible values: openai, groq, deepgram, fastwhisperapi
    RESPONSE_MODEL = 'openai'  # !CHANGED THIS TO OPENAI.
    TTS_MODEL = 'deepgram'  # possible values: openai, deepgram, elevenlabs, melotts, cartesia

    # currently using the MeloTTS for local models. here is how to get started:
    # https://github.com/myshell-ai/MeloTTS/blob/main/docs/install.md#linux-and-macos-install

    # LLM Selection
    OLLAMA_LLM="llama3:8b"
    GROQ_LLM="llama3-8b-8192"
    #OPENAI_LLM="gpt-4o"
    OPENAI_LLM="llama3-8b-8192" # !CHANGED THE MODEL HERE TO A GROQ-MODEL.

    # API keys and paths
    #OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
    OPENAI_API_KEY = os.getenv("GROQ_API_KEY") # !ENTERED THE GROQ-API-KEY AS THE OPENAI-KEY.
    GROQ_API_KEY = os.getenv("GROQ_API_KEY")
    DEEPGRAM_API_KEY = os.getenv("DEEPGRAM_API_KEY")
    ELEVENLABS_API_KEY = os.getenv("ELEVENLABS_API_KEY")
    LOCAL_MODEL_PATH = os.getenv("LOCAL_MODEL_PATH")
    CARTESIA_API_KEY = os.getenv("CARTESIA_API_KEY")

This way I don't use the GROQ-Module. I use the OpenAI-Module in compatibility-mode from Groq.

I am currently using Python 3.12.3.

requirements.txt

deepgram-sdk==3.2.7
groq==0.8.0
openai==1.30.1

Maybe something wrong in the Groq-Module version I am using.

It works this way. I still have no idea why it does not work with the Groq module.

[PromtEngineer]

@mmuyakwa seems like since everyone is following the Openai standard, it might be just easier to move to use the openai client like you did the integration and just provide the base_url. even google is doing that now with gemini.