This allows to limit max_tokens used for a request.
This also fixes using shell-ai with Mistral AI.
Mistral AI is supposed to have the same API as OpenAI but unfortunately setting up max_tokens=null in a request makes the request fail with error 500.
This change allows to use Mistral AI API, before they fix it.
ricklamers
commented
11 months ago
This allows to limit max_tokens used for a request.
This also fixes using shell-ai with Mistral AI. Mistral AI is supposed to have the same API as OpenAI but unfortunately setting up
max_tokens=nullin a request makes the request fail with error 500.This change allows to use Mistral AI API, before they fix it.