Open cfernhout opened 2 days ago
Thanks for reporting — I can reproduce the ValidationError
. This is coming from the Converse API, so needs to be supported upstream by the Bedrock service.
Upon searching internally, I found that the Bedrock team is aware of this and has plans to increase the maxTokens
validation number accordingly. You can follow the blog and CHANGELOG for updates.
(For now if you remove the inferenceConfig
from that snippet or specify <= 4096 then it should work. I tested with modelId="anthropic.claude-3-5-sonnet-20240620-v1:0"
and the request succeeded after doing that.)
Describe the bug
When invoking converse with maxTokens > 4096, it tells me that I cant. But Clause Sonnet 3.5 has a token limit of 8192. I've tested to use the anthropic SDK instead of boto3 and that works.
Expected Behavior
Should work with maxTokens up to 8192.
Current Behavior
Reproduction Steps
Possible Solution
No response
Additional Information/Context
No response
SDK version used
1.35.2
Environment details (OS name and version, etc.)
Linux and MacOs.