Prompt Engineering | Prompt Versioning | Use GPT or other prompt based models to get structured output. Join our discord for Prompt-Engineering, LLMs and other latest research
Add suffix parameter to GPT Completion models (not available for ChatGPT)
Add all other parameters to ChatCompletion models (stop, top_p, frequency_penalty, presence_penalty)
Automatically determine max output tokens based on GPT model if max_tokens not specified
Currently, the user specifies max total tokens, and then max output tokens is automatically determined by deducting the prompt tokens. This is unintuitive because
The user shouldn't have to keep track of the max total tokens for a given model
The max_tokens parameter in OpenAI models refers to the max output tokens, not max total, so we're using the phrase ambiguously here
I suggest the following, depending on whether user specifies max_tokens:
If unspecified (max_tokens=None), automatically determine max total tokens based on model name and deduct prompt tokens
If specified, simply use max_tokens as the max output tokens (aligning with how the OpenAI model uses it).
I apologize I also shuffled some other stuff around for vanity, and now it's hard to undo. I can remove those superfluous changes if you prefer.
Three changes to OpenAI model parameters
suffix
parameter to GPT Completion models (not available for ChatGPT)stop
,top_p
,frequency_penalty
,presence_penalty
)max_tokens
not specifiedmax_tokens
parameter in OpenAI models refers to the max output tokens, not max total, so we're using the phrase ambiguously heremax_tokens
:max_tokens=None
), automatically determine max total tokens based on model name and deduct prompt tokensmax_tokens
as the max output tokens (aligning with how the OpenAI model uses it).I apologize I also shuffled some other stuff around for vanity, and now it's hard to undo. I can remove those superfluous changes if you prefer.