aws-samples / bedrock-claude-chat

AWS-native chatbot using Bedrock + Claude (+Mistral)
MIT No Attribution
908 stars 324 forks source link

feat: Cross region inference for v2 #577

Closed axelpina closed 2 weeks ago

axelpina commented 4 weeks ago

Issue #535, if available:

Description of changes:

šŸšØ NOTE: Accidentally erased the previous PR while trying to pull the changes from the most recent commits. šŸ™šŸ½

This new PR addresses the points made by @statefb

Previous PR: https://github.com/aws-samples/bedrock-claude-chat/pull/571

utils.py

  1. Added ENABLE_BEDROCK_CROSS_REGION_INFERENCE which determines if cross-region inference is enabled.
  2. Added SUPPORTED_REGIONS which is a constant list of regions that support cross-region inference.
  3. Renamed is_region_supported_for_inference to is_region_supported_for_cross_inference(region: str) to emphasize its purpose for cross-region inference.
  4. Modified get_bedrock_runtime_client to add logic which checks if cross-region inference is enabled and if the region is supported. If both conditions are met, it initializes the Bedrock Runtime client for the specified region. If not, it logs a warning and defaults to using BEDROCK_REGION.
  5. Ensured BEDROCK_REGION is used consistently aligning with @statefb's comments in feat: Add cross-region inference support for Bedrock models (#535) #536

bedrock.py

Credit to @chm10 for the initial implementation in version v1.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

statefb commented 2 weeks ago

@axelpina Due to high demand, I've taken the initiative to implement this feature myself. However, I sincerely appreciate your efforts and willingness to tackle this! Thank you for your understanding.