bug: generation max token limits

IBM / ibm-generative-ai

IBM-Generative-AI is a Python library built on IBM's large language model REST interface to seamlessly integrate and extend this service in Python programs.

https://ibm.github.io/ibm-generative-ai/

Apache License 2.0

245 stars 101 forks source link

bug: generation max token limits #332

Closed Tomas2D closed 5 months ago

Tomas2D commented 6 months ago

Currently generated Pydantic model has max_new_tokens set to 4096. This value is not valid anymore and has been recently removed from the OpenAPI schema. We want to regenerate the schema and remove such limitations.

Workaround

from genai.schema import TextGenerationParameters

TextGenerationParameters(max_new_tokens=8000) # ❌
TextGenerationParameters.model_construct(max_new_tokens=8000) # ✅

Tomas2D commented 5 months ago

Released in version 2.3.0.