AWS Bedrock provider doesn't work unless using `bedrockOptions`

elie222 commented 3 weeks ago

Description

This works:

createAmazonBedrock({
          bedrockOptions: {
            region: env.BEDROCK_REGION,
            credentials: {
              accessKeyId: env.BEDROCK_ACCESS_KEY,
              secretAccessKey: env.BEDROCK_SECRET_KEY,
            },
          },
        })(model)

This works when running locally, but not when deploying to Vercel:

createAmazonBedrock({
            region: env.BEDROCK_REGION,
              accessKeyId: env.BEDROCK_ACCESS_KEY,
              secretAccessKey: env.BEDROCK_SECRET_KEY,
          },
        })(model)

I imagine the issue is that Vercel is setting a key that causes the error (https://vercel.com/docs/projects/environment-variables/reserved-environment-variables#allowed-environment-variables). But I didn't manage to track down the error. bedrockOptions worked for me though.

Code example

No response

Additional context

No response

shaper commented 3 weeks ago

Notes while investigating though I don't have a proposed fix yet:

I can reproduce the issue. As you suggest, it looks like an AWS_SESSION_TOKEN env var is set in the deployment but not when running locally. This is used by AI SDK logic as the default value for sessionToken when instantiating the AWS BedrockRuntimeClient.

Both of your examples (with the noted typo below fixed) work for me locally. In a deployment, the second one seems to execute one query successfully but subsequent queries fail, perhaps due to re-use of a session token intended for refresh (though passing the session token along at all is unintended AFAIK and so part of the bug).

~~I'm not sure what's setting the env var: Vercel's containing environment, the AWS lib, or something else.~~ A clean app without anything AWS installed exhibits the env var set when a serverless function executes, so I think it's the Vercel environment.

Side note -- there is an extra closing curly brace in your doesn't-work example above. It looks like a typo from removing the credentials sub-object. Here is the corrected second example:

createAmazonBedrock({
  region: env.BEDROCK_REGION,
  accessKeyId: env.BEDROCK_ACCESS_KEY,
  secretAccessKey: env.BEDROCK_SECRET_KEY,
})(model)

shaper commented 3 weeks ago

For an AI SDK fix, I am wondering whether perhaps we should remove the fallback to AWS_XXX vars in this provider, since as it is we risk clashing with values set by an external runtime that differ from what the user intends. The basic problem is that one typically has some creds one wants to use for interacting with Bedrock (say creds set A), but here we're running in a serverless function that has a bunch of env vars already set (say creds set B) and we can unintentionally partially merge the two.

It's only the token that's problematic currently, but if we special-case it (don't use its value as a fallback since it can be set outside of the user's control) but leave the others, then we have an API that could surprise a developer.

We could prefix the default env var names with AI_SDK_ to avoid conflict, but this would be the first provider where we'd be doing that.

elie222 commented 2 weeks ago

For an AI SDK fix, I am wondering whether perhaps we should remove the fallback to AWS_XXX vars in this provider, since as it is we risk clashing with values set by an external runtime that differ from what the user intends. The basic problem is that one typically has some creds one wants to use for interacting with Bedrock (say creds set A), but here we're running in a serverless function that has a bunch of env vars already set (say creds set B) and we can unintentionally partially merge the two.

It's only the token that's problematic currently, but if we special-case it (don't use its value as a fallback since it can be set outside of the user's control) but leave the others, then we have an API that could surprise a developer.

We could prefix the default env var names with AI_SDK_ to avoid conflict, but this would be the first provider where we'd be doing that.

Yes, I agree it should be renamed as it's already in use by Vercel for other purposes. And many people using this library will encounter that issue.

vercel / ai