This commit introduces the initial setup for supporting cross-region inference in the Bedrock Chat application.
Issue #, if available:
535
Description of changes:
Added is_region_supported_for_inference function in utils.py to check if a region supports inference.
Modified get_bedrock_client function in utils.py to use cross-region inference if enabled and the region is supported.
Updated get_model_id function in bedrock.py to use the base model ID for cross-region inference if enabled, the region is supported, and the model is included in CROSS_REGION_INFERENCE_MODELS. If any of these conditions are not met, it falls back to using the local model ID and logs a warning.
Added enableBedrockCrossRegionInference option to cdk.json with the default value set to false.
These changes lay the foundation for enabling cross-region inference in the Bedrock Chat application. The feature can be enabled or disabled using the enableBedrockCrossRegionInference configuration option in cdk.json.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
@chm10 Due to high demand, I've taken the initiative to implement this feature myself (#583 ). However, I sincerely appreciate your efforts and willingness to tackle this! Thank you for your understanding.
This commit introduces the initial setup for supporting cross-region inference in the Bedrock Chat application.
Issue #, if available:
535
Description of changes:
is_region_supported_for_inference
function inutils.py
to check if a region supports inference.get_bedrock_client
function inutils.py
to use cross-region inference if enabled and the region is supported.get_model_id
function inbedrock.py
to use the base model ID for cross-region inference if enabled, the region is supported, and the model is included inCROSS_REGION_INFERENCE_MODELS
. If any of these conditions are not met, it falls back to using the local model ID and logs a warning.enableBedrockCrossRegionInference
option tocdk.json
with the default value set tofalse
.These changes lay the foundation for enabling cross-region inference in the Bedrock Chat application. The feature can be enabled or disabled using the
enableBedrockCrossRegionInference
configuration option incdk.json
.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.