search

groq / groq-python

The official Python Library for the Groq API
Apache License 2.0
353 stars 23 forks source link

Is there a way to adjust the logit likliehood using the groq api #31

Closed psych0v0yager closed 3 months ago

psych0v0yager commented 4 months ago

I am planning to add groq support into the constrained generation library outlines. https://github.com/outlines-dev/outlines Outlines already has OpenAI support so the change should be very straightforward except for the following issue

The documentation for groq says the following features are not supported in the groq api

logprobs logit_bias top_logprobs

https://console.groq.com/docs/openai

Is there an alternative way to adjust the logit probabilities in groq or is it currently not supported?

gradenr commented 3 months ago

This is on the roadmap, but is not currently available with our API. I have shared this requests with our product team.