Exposing logit_bias at the raw API will unblock customers who need direct access; however, managing tiers of logit_bias can be challenging so additional tooling around this could be useful in the future.
One thing to note is that all models don't support the same properties and OpenAI is the main one that supports logit_bias. Thus, thought will need to be put into how model specific properties are exposed.
chipalost
commented
1 year ago
Exposing logit_bias at the raw API will unblock customers who need direct access; however, managing tiers of logit_bias can be challenging so additional tooling around this could be useful in the future.
One thing to note is that all models don't support the same properties and OpenAI is the main one that supports logit_bias. Thus, thought will need to be put into how model specific properties are exposed.