Open syberkitten opened 5 months ago
The max Output Tokens for Gemini models are 8192. See here: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini
When selecting Gemini models, Big-AGI lets you change the output tokens up to the full range. The values we get are directly out of the Gemini models list API, and exceeding those values will return an error.
Just tested that for Gemini 1.5 Pro, I can select from 256 to 8192 (the full max) supported.
In your case, just update the models, go to the model options (wheel icon) and drag the token window to the max.
I see the usefulness of this feature, however the real issue is that some APIs do not report the context window of the models, or the maximum output tokens.
Anthropic, OpenaAI, Google and OpenRouter either have the values in the APIs (preferred) or the web page.
I'll take a look at implementing but there are 2 issues with not selecting the output size:
This is the primary reason why I'm currently exploring other LLM front-ends. 2048 max is much too little for many models and significantly lowers the usability of many when you've run out of tokens and forced to compress or restart. Great for single prompt response, not so much for deeper analysis and contextual dialog. Please enable to at least push to 8192 or even 32768.
See #531
Why
Description
by default don't cap models to 2048 - uncap. But if you need to cap, at least allow to move it further (currently I cannot move gemini more then 2048, rediculous)