Cross-region support for Bedrock

[shabie]

Aamzon Bedrock now supports cross-region inference making it easier to handle throughput: https://aws.amazon.com/about-aws/whats-new/2024/08/amazon-bedrock-cross-region-inference/

This requires using a model ID with the region prefix as can be seen here:

Inference profile	Inference profile ID	Regions included
US Anthropic Claude 3 Haiku	us.anthropic.claude-3-haiku-20240307-v1:0	US East (N. Virginia) (us-east-1) US West (Oregon) (us-west-2)
EU Anthropic Claude 3 Haiku	eu.anthropic.claude-3-haiku-20240307-v1:0	Europe (Frankfurt) (eu-central-1) Europe (Ireland) (eu-west-1) Europe (Paris) (eu-west-3)
US Anthropic Claude 3 Opus	us.anthropic.claude-3-opus-20240229-v1:0	US East (N. Virginia) (us-east-1) US West (Oregon) (us-west-2)
US Anthropic Claude 3 Sonnet	us.anthropic.claude-3-sonnet-20240229-v1:0	US East (N. Virginia) (us-east-1) US West (Oregon) (us-west-2)
EU Anthropic Claude 3 Sonnet	eu.anthropic.claude-3-sonnet-20240229-v1:0	Europe (Frankfurt) (eu-central-1) Europe (Ireland) (eu-west-1) Europe (Paris) (eu-west-3)
US Anthropic Claude 3.5 Sonnet	us.anthropic.claude-3-5-sonnet-20240620-v1:0	US East (N. Virginia) (us-east-1) US West (Oregon) (us-west-2)
EU Anthropic Claude 3.5 Sonnet	eu.anthropic.claude-3-5-sonnet-20240620-v1:0	Europe (Frankfurt) (eu-central-1) Europe (Ireland) (eu-west-1) Europe (Paris) (eu-west-3)

This means that this code needs to be modified:

https://github.com/langchain-ai/langchain-aws/blob/d89fcf80c61174c21c88ef8d1cfbf3a55c7d84c5/libs/aws/langchain_aws/llms/bedrock.py#L616-L625

This logic should be updated to something like the following:

def _get_provider(self) -> str:
    if self.provider:
        return self.provider
    if self.model_id.startswith("arn"):
        raise ValueError(
            "Model provider should be supplied when passing a model ARN as "
            "model_id"
        )

    first_element, second_element, *_ = self.model_id.split(".")
    if first_element.lower() in {"eu", "us", "ap", "sa"}:
        return second_element
    return first_element

I am happy to make a PR.

[rsgrewal-aws]

@3coins and @shabie can we please urgently get this fix in - this is needed for the Custom Model import feature of Bedrock as well

for now will sub-class work ?

[rsgrewal-aws]

what is the purpose of this code -

if self.model_id.startswith("arn"): raise ValueError( "Model provider should be supplied when passing a model ARN as " "model_id" )

[kai-zhu-aws]

https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference-support.html Seems only us and eu are supported right now.

[kai-zhu-aws]

There is a PR for it https://github.com/langchain-ai/langchain-aws/pull/170/files

[3coins]

https://github.com/langchain-ai/langchain-aws/pull/184 fixes this issue. @shabie Can you take a look, there is a slight modification from your code suggestion to allow some of the test runs to pass.