Allow changing context length per chat session

[AriMKatz]

Validations

• [X] I believe this is a way to improve. I'll try to join the Continue Discord for questions
• [X] I'm not able to find an open issue that requests the same enhancement

Problem

Newer models like sonnet 3.5 do well across almost the entire context window, but I might not want to use it for every chat.

Solution

Please allow dynamically setting context window.

Right now only can change in model load https://github.com/continuedev/continue/issues/1364

Also, does the context provider account for this?

[sestinj]

@AriMKatz The best way to solve this is probably to make two separate copies of your model, each with a different "contextLength" parameter

After context providers resolve their contents, we then do prompt pruning to fit it into the context window, so context providers like @ codebase don't take into account contextLength by themselves

Seems like what you might want is the ability to have @ codebase automatically grab more results when you're using the longer context model?

[AriMKatz]

Model copying sounds fine.

"

Seems like what you might want is the ability to have @ codebase automatically grab more results when you're using the longer context model?

Yes exactly