Add groq API support

[jeffconrad18]

Please add support for the API of groq. It is using the same syntax as OpenAI but is much faster and allows the user to use Open Source models.

` import os

from groq import Groq

client = Groq( api_key=os.environ.get("GROQ_API_KEY"), )

chat_completion = client.chat.completions.create( messages=[ { "role": "user", "content": "Explain the importance of fast language models", } ], model="mixtral-8x7b-32768", )

print(chat_completion.choices[0].message.content) `

[AndraxDev]

Can't you just set custom host to Groq (I don't know it but you might) and then set an Groq key inside OpenAI key dialog? API key is just saved and not validated through regex. So technically app accepts any text as API key unless it empty.

You can set custom API host in global settings:

[AndraxDev]

"OpenAI" word will be removed in future to not confuse users so this app supports OpenAI only. This app uses this AI library: https://github.com/aallam/openai-kotlin

Please refer it's documentation to find if Groq is fully supported.

Thanks!

[AndraxDev]

Bug found: Fine-tuned models does not use chat completions and non-gpt models are threated as fine-tuned. I must fix some regex patterns.

But in overall I found that custom API host is working and API is also working as I sucessfully obtained list of models.

[jeffconrad18]

Great, groq is a great upgrade, it's so fast and free as well.

[AndraxDev]

Starting from SpeakGPT 3.17 you cat use GROQ models. Please check your API host if models don't work. Correct API host: https://api.groq.com/openai/v1/

To select a GROQ models open model selection dialog > See all models. If hostname is correct you will see something like this:

[AndraxDev]

Play Store releases usually delayed for 1-2 business days because of moderations.

[jeffconrad18]

When using groq I first get an answer but then I always get the following error: Looks like this model (mixtral-8x7b-32768) is not available to you right now. It can be because of high demand or this model is currently in limited beta. If you are using a fine-tuned model, please make sure you entered correct model name. Usually model starts with 'model_name:ft-' and contains original model name, organization name and timestamp. Example: ada:ft-organization_name:model_name-YYYY-MM-DD-hh-mm-ss.

[AndraxDev]

Now I verified that it's working. Please check your input for misspells, and use advanced models selector (Settings > Models > See all models and then select the model). Also make sure API host is set correctly. Should: https://api.groq.com/openai/v1/ (!!!NOT https://api.groq.com/openai/v1/chat/ o or https://api.groq.com/openai/v1/completions).

[AndraxDev]

Now I verified that it's working. Please check your input for misspells, and use advanced models selector (Settings > Models > See all models and then select the model). Also make sure API host is set correctly. Should: https://api.groq.com/openai/v1/ (!!!NOT https://api.groq.com/openai/v1/chat/ o or https://api.groq.com/openai/v1/completions).

[jeffconrad18]

I've already done what you just wrote me but after the answer there still comes this error message: Looks like this model (mixtral-8x7b-32768) is not available to you right now. It can be because of high demand or this model is currently in limited beta. If you are using a fine-tuned model, please make sure you entered correct model name. Usually model starts with 'model_name:ft-' and contains original model name, organization name and timestamp. Example: ada:ft-organization_name:model_name-YYYY-MM-DD-hh-mm-ss.

[AndraxDev]

Please login at groq and veify if you still have access to this model (if trial period is still available to you). This message is displayed only when server returned 404 error. If SpeakGTp encounters in a different problems it will display stacktrace instead.

[AndraxDev]

Also please make sure you have updated an app to the latest version (3.17)

[jeffconrad18]

It's basically working but then this error appears.