This pull request includes a minor update to the prepare_body method in the amazon_bedrock integration. The change ensures compatibility with Anthropic by removing the unsupported stream keyword argument from inference_kwargs.
We need to leave "stream" kwarg as some other models like Cohere do require it and other LLM foundations models are able to tolerate its presence. We'll soon completely rewrite entire AmazonBedrockChatGenerator codebase to use Converse API anyway.
Note: I didn’t add any new tests but ran the existing ones on CI and locally (for Mistral) to review the logs. While adding new tests would typically be necessary for these fixes I skipped them in this case because we’re planning a complete rewrite of the Amazon Bedrock integration soon anyway.
This pull request includes a minor update to the
prepare_bodymethod in theamazon_bedrockintegration. The change ensures compatibility with Anthropic by removing the unsupportedstreamkeyword argument frominference_kwargs.We need to leave "stream" kwarg as some other models like Cohere do require it and other LLM foundations models are able to tolerate its presence. We'll soon completely rewrite entire AmazonBedrockChatGenerator codebase to use Converse API anyway.
Note: I didn’t add any new tests but ran the existing ones on CI and locally (for Mistral) to review the logs. While adding new tests would typically be necessary for these fixes I skipped them in this case because we’re planning a complete rewrite of the Amazon Bedrock integration soon anyway.