Cost Effective Model Selection

[mfbayraktar]

In general, I typically utilize a 4K context. However, there are cases where I require a longer context of 16 tokens. In such cases, I have two options:

1. Opt for a model with a larger context limit (16K) without the need to change the model name every time. This ensures I can consistently send/receive longer outputs, albeit at a higher cost.
2. Stick with the 4K context and switch to a different model when a longer context is needed. While this option is more cost-effective, it does require a bit of additional time to make the model change.

The system provides an error message indicating the context limit (e.g., "Error: This model's maximum context length is 4097 tokens. However, you requested X tokens (Y in the messages, Z in the completion). Please reduce the length of the messages or completion.").

Based on this message or through automated calculation, the system can select the alternative model automatically.

[rolandcucicea]

You could set up a keyboard macro to change the model, not sure how other platforms work but on macos with keyboard maestro/ apple shortcuts you could set up a macro that on a high level: opens text file searches for the model field deletes the text replaces with wanted model

u can have two macros, one for each model. Doing this programatically based on the api response could incur unwated costs to other users (personally I'm mostly using 3.5turbo beacuse it's cheap and good enough for me). Plus it's not failsafe, openai updates their api, so error message might not match a hard coded string in future updates