Added ENABLE_BEDROCK_CROSS_REGION_INFERENCE which determines if cross-region inference is enabled.
Added SUPPORTED_REGIONS which is a constant list of regions that support cross-region inference.
Renamed is_region_supported_for_inference to is_region_supported_for_cross_inference(region: str) to emphasize its purpose for cross-region inference.
Modified get_bedrock_runtime_client to add logic which checks if cross-region inference is enabled and if the region is supported. If both conditions are met, it initializes the Bedrock Runtime client for the specified region. If not, it logs a warning and defaults to using BEDROCK_REGION.
Ensured BEDROCK_REGION is used consistently aligning with @statefb's comments in feat: Add cross-region inference support for Bedrock models (#535) #536
bedrock.py
Imported is_region_supported_for_cross_inference and ENABLE_BEDROCK_CROSS_REGION_INFERENCE from utils.py
Adjusted get_model_id function.
Moved CROSS_REGION_INFERENCE_MODELS inside the get_model_id function, as it's only used there.
Added logic to check if cross-region inference is enabled, the region is supported, and the model supports cross-region inference.
If all conditions are met, returns the base model ID for cross-region inference.
If not, logs appropriate warnings and adjusts the model ID to use the local model. Included comments to explain the adjustments made to model IDs for local inference, addressing the concern about prefixes and suffixes
Ensured BEDROCK_REGION is used throughout for Bedrock services.
Credit to @chm10 for the initial implementation in version v1.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
@axelpina Due to high demand, I've taken the initiative to implement this feature myself. However, I sincerely appreciate your efforts and willingness to tackle this! Thank you for your understanding.
Issue #535, if available:
Description of changes:
šØ NOTE: Accidentally erased the previous PR while trying to pull the changes from the most recent commits. šš½
This new PR addresses the points made by @statefb
Previous PR: https://github.com/aws-samples/bedrock-claude-chat/pull/571
utils.py
ENABLE_BEDROCK_CROSS_REGION_INFERENCE
which determines if cross-region inference is enabled.SUPPORTED_REGIONS
which is a constant list of regions that support cross-region inference.is_region_supported_for_inference
tois_region_supported_for_cross_inference(region: str)
to emphasize its purpose for cross-region inference.get_bedrock_runtime_client
to add logic which checks if cross-region inference is enabled and if the region is supported. If both conditions are met, it initializes the Bedrock Runtime client for the specified region. If not, it logs a warning and defaults to usingBEDROCK_REGION
.BEDROCK_REGION
is used consistently aligning with @statefb's comments in feat: Add cross-region inference support for Bedrock models (#535) #536bedrock.py
ENABLE_BEDROCK_CROSS_REGION_INFERENCE
from utils.pyCROSS_REGION_INFERENCE_MODELS
inside theget_model_id
function, as it's only used there.BEDROCK_REGION
is used throughout for Bedrock services.Credit to @chm10 for the initial implementation in version v1.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.