search

adamrushy / OpenAISwift

This is a wrapper library around the ChatGPT and OpenAI HTTP API
MIT License
1.6k stars 242 forks source link

Add support for `logit_bias` and improve documentation #51

Closed julianschiavo closed 1 year ago

julianschiavo commented 1 year ago

See OpenAI documentation on logit_bias:

Modify the likelihood of specified tokens appearing in the completion.

Accepts a json object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.

adamrushy commented 1 year ago

Nice improvements 👏🏼 appreciate the contribution!

julianschiavo commented 1 year ago

Happy to help, thank you for the library! It's been super useful.