When a user selects some of the latest OpenAI models (given that they have been given access to them by OpenAI), the following warning will appear "Warning: Error while counting tokens - skipping token limit checks".
This is likely to be resolved with an update of the tiktoken dependency - or, in the worst case, a fallback to the next best approximation for counting tokens.
When a user selects some of the latest OpenAI models (given that they have been given access to them by OpenAI), the following warning will appear "Warning: Error while counting tokens - skipping token limit checks".
This is likely to be resolved with an update of the tiktoken dependency - or, in the worst case, a fallback to the next best approximation for counting tokens.