search

e-p-armstrong / augmentoolkit

Convert Compute And Books Into Instruct-Tuning Datasets
MIT License
584 stars 79 forks source link

Request to add Groq API #14

Closed KlausikPL closed 1 month ago

KlausikPL commented 2 months ago

Would it be possible to add Groq AI support? It's free and extremely fast.Screenshot_2024-05-04-17-41-56-95_40deb401b9ffe8e1df2f1cc5ba480b12.jpg

e-p-armstrong commented 1 month ago

Groq is openai compatible. You should just be able to change the base URL and Augmentoolkit will probably work with it (note that Llama 3 does not have enough context for the final step if it's not in assistant mode).

The how-to is in their documentation here https://console.groq.com/docs/openai, but basically you just change the base_url in the config file to https://api.groq.com/openai/v1

KlausikPL commented 1 month ago

Thanks you

KlausikPL commented 1 month ago

I have one more question, can I use the cohere API because I tried it and nothing but groq works normally?

e-p-armstrong commented 1 month ago

cohere has its own SDK. I actually have some experimental projects where cohere's added to the engine wrapper, I'll look into adding it back into the OG Augmentoolkit.

KlausikPL commented 1 month ago

any update about Cohere?