search

outlines-dev / outlines

Structured Text Generation
https://outlines-dev.github.io/outlines/
Apache License 2.0
6.98k stars 359 forks source link

Groq Integration #946

Open rv4102 opened 3 weeks ago

rv4102 commented 3 weeks ago

Since Groq follows OpenAI API, would it be possible to add support for it in the library?

sharanry commented 1 week ago

As far as I understand, this might not be possible as Groq OpenAI API interface doesn't support logprobs:

Groq Cloud Documentation (https://console.groq.com/docs/openai):

image

@rlouf am I correct in my understanding?