search

aws-amplify / amplify-flutter

A declarative library with an easy-to-use interface for building Flutter applications on AWS.
https://docs.amplify.aws
Apache License 2.0
1.31k stars 242 forks source link

[aws_signature_v4] Expose service configuration for Bedrock #5236

Open dkliss opened 1 month ago

dkliss commented 1 month ago

Description

Hi,

I am having problems in using aws_sig4. What I am doing is,

  1. Login via AWS cognito.
  2. Get user credentials via final result = await cognitoPlugin.fetchAuthSession();
  3. Setup a button to initiate function _initializeClient() and pass credentials attained by above function.
  4. My endpoint: bedrock-runtime.us-east-1.amazonaws.com

ISSUE

I am continually getting error "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details"

The CODE FAILS with response code 403. Key function is below. Is there anything I need to fix below or any misunderstanding on my behalf. Please let me know.

Packages Used

aws_common: ^0.7.1
aws_client: ^0.6.0
aws_signature_v4: ^0.6.1
amplify_flutter: ^2.3.0
amplify_auth_cognito: ^2.3.0

Reference function: 


 Map<String, String> _getCommonHeaders() {
    return {
      AWSHeaders.contentType: 'application/json', // Set content type as JSON
      AWSHeaders.accept: 'application/json',
      AWSHeaders.authorization: '', 
      // AWSHeaders.authorization:
      //     'Bearer ${authCredentials.sessionTokens.idToken}', // Use the ID token for authorization
      AWSHeaders.securityToken: authCredentials.sessionTokens.sessionToken ??
          '', // Use the session token
      'X-Amzn-Bedrock-GuardrailIdentifier':
          guardRailIdentifier, // Custom header value for guardrail identifier
      'X-Amzn-Bedrock-GuardrailVersion': '1', // Version for the guardrail
      'X-Amzn-Bedrock-Trace': 'trace', // Optional tracing header
    };
  }

Future<void> _initializeClient() async {
    try {
      // Define AWS service
      const service = common.AWSService('bedrock');

      // Create AWS credentials
      final credentials = AWSCredentialsProvider(AWSCredentials(
        authCredentials.sessionTokens.accessKeyID,
        authCredentials.sessionTokens.secretAccessKey,
        authCredentials.sessionTokens.sessionToken,
        authCredentials.sessionTokens.expiry,
      ));

      // Initialize the signer
      signer = AWSSigV4Signer(
        credentialsProvider: credentials,
        algorithm: AWSAlgorithm.hmacSha256,
      );

      final scope = AWSCredentialScope(
        region: region,
        service: service,
      );

      // Create the AWSBaseHttpRequest
      final awsRequest = common.AWSHttpRequest(
        method: common.AWSHttpMethod.post,
        uri: //Uri.https('$bedrockEndpoint/model/$modelId/invoke'),
            Uri.https(
          bedrockEndpoint, // The hostname
          '/model/$modelId/invoke', // The path
        ),
        headers: _getCommonHeaders(),
        body: json.encode({
          'amazon-bedrock-guardrailConfig': {
            'tagSuffix': 'string', // Replace with actual tag suffix if needed
          },
        }).codeUnits,
      );

      // Sign the request
      final signedRequest = await signer.sign(
        awsRequest,
        credentialScope: scope,
      );

      debugPrint(
          'Response from AWS Bedrock HTTP Method: ${signedRequest.canonicalRequest}');
      debugPrint('Response from AWS Bedrock Path: ${signedRequest.path}');
      debugPrint(
          'Response from AWS Bedrock queryParameters: ${signedRequest.queryParameters}');
      debugPrint(
          'Response from AWS Bedrock queryParametersAll: ${signedRequest.queryParametersAll}');

      debugPrint('Response from AWS Bedrock host: ${signedRequest.host}');
      debugPrint(
          'Response from AWS Bedrock hasContentLength: ${signedRequest.hasContentLength}');
      debugPrint('Response from AWS Bedrock headers: ${signedRequest.headers}');
      // Send the request using AWSHttpClient
      final operation = signedRequest.send();

      // Handle the response
      final respBody = await operation.response;

      debugPrint('Response from AWS Bedrock: ${respBody.statusCode}');

      debugPrint(
          'Response from AWS Bedrock: ${await processResponse(respBody.body)}');

      if (respBody.statusCode != 200) {
        throw HttpRequestException(
          message: 'Request failed: ${respBody.statusCode}',
          statusCode: respBody.statusCode,
        );
      }

      // Initialize _bedrockClient with the correct client instance
      _bedrockClient = bedrock.BedrockRuntime(
        //client: client as http.Client,
        region: region,
        credentials: bedrock.AwsClientCredentials(
          accessKey: authCredentials.sessionTokens.accessKeyID,
          secretKey: authCredentials.sessionTokens.secretAccessKey,
          sessionToken: authCredentials.sessionTokens.sessionToken,
          expiration: authCredentials.sessionTokens.expiry,
        ),
        endpointUrl: bedrockEndpoint,
      );
    } catch (e) {
      debugPrint('Authentication request failed: $e');
      // Optionally rethrow or handle the error as needed
      throw HttpRequestException(
        message: 'Failed to initialize client: $e',
      );
    } finally {
      //client.cclose(); // Close the client when done
    }
  }

Categories

Steps to Reproduce

  1. Login via AWS cognito.
  2. Get user credentials via final result = await cognitoPlugin.fetchAuthSession();
  3. Setup a button to initiate function _initializeClient() and pass credentials attained by above function.
  4. My endpoint: bedrock-runtime.us-east-1.amazonaws.com

Screenshots

NA

Platforms

Flutter Version

latest

Amplify Flutter Version

2.3.0

Deployment Method

Amplify CLI

Schema

NA
Equartey commented 1 month ago

Hi @dkliss, thanks for taking the time to submit this.

We will need to try to reproduce the observed behavior.

Can you please provide the full error message and relevant logs?

dkliss commented 1 month ago

hi @Equartey

The Error Message is simply below". The error occurs specifically for due to function final operation = signedRequest.send();. The status code received from this function call is 403. And the credentials which are used in this request is attained from await cognitoPlugin.fetchAuthSession().

"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."

If i simply run below without using aws_client: ^0.6.0 &. aws_signature_v4: ^0.6.1 packages, then I still get the same error

" Network error: Authentication request failed: 403 InvalidSignatureException: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details" 

  Future<void> authenticateUser(AuthCredentials newAuthCredentials) async {
    try {
      // Reinitialize the Bedrock client with new credentials
      //await _initializeClient(); // Ensure _bedrockClient is initialized
      _bedrockClient = bedrock.BedrockRuntime(
        //client: client as http.Client,
        region: region,
        credentials: bedrock.AwsClientCredentials(
          accessKey: authCredentials.sessionTokens.accessKeyID,
          secretKey: authCredentials.sessionTokens.secretAccessKey,
          sessionToken: authCredentials.sessionTokens.sessionToken,
          expiration: authCredentials.sessionTokens.expiry,
        ),
        endpointUrl: bedrockEndpoint,
      );

    } catch (e) {
      throw HttpRequestException(
        message: 'Authentication request failed: $e',
      );
    }
  }
Equartey commented 1 month ago

Hi @dkliss, thanks for providing that extra context. We will get back to you with our findings.

dkliss commented 1 month ago

Hi @Equartey , I was wondering if any progress on this or if this is still in backlog?

Equartey commented 4 weeks ago

Hi @dkliss, apologies for the delay, we have not got to this yet.

Out of curiosity are you able to make requests to other endpoints using your setup?

dkliss commented 4 weeks ago

Hi @Equartey Thanks for your response.

In us-east-1, the only endpoints listed are below below

  static const bedrockEndpointControl = 'bedrock.us-east-1.amazonaws.com';
  static const bedrockEndpointRuntime =
      'bedrock-runtime.us-east-1.amazonaws.com';

I tried changing mode though from haiku to an aws internal model below using the same approach i mentioned above but no luck.

static const modelId = 'amazon.titan-text-lite-v1';

I then attempted a direct http request but got credential error which i suspect is because of not using aws-sig4.

I am bit unclear if I am missing a key configuration here or am passing correct keys. I expect the authentications keys from await cognitoPlugin.fetchAuthSession(); to authentication client request to AWS but something seem to be not allowing this (403 error always). Seems like aws_sig4 but am not able to get to a conclusion here. So any help is appreciated. 

Specs: Http request with bedrockEndpoint: bedrock-runtime.us-east-1.amazonaws.com and modeID: amazon.titan-text-lite-v1'

ERROR from below http request received: 

flutter: Response from aws bedrock: {"message":"Invalid key=value pair (missing equal-sign) in Authorization header (hashed with SHA-256 and encoded with Base64): '*************************************************='."}
flutter: Response from aws bedrock: 403

// Authenticate directly via http request
  Future<http.Response> authenticateUserviaHttp() async {
    final url = Uri.parse('https://$bedrockEndpoint/model/$modelId/invoke');

    final response = await http.post(
      url,
      headers: _getCommonHeaders(),
      body: jsonEncode({
        'amazon-bedrock-guardrailConfig': {
          'tagSuffix': 'string' // Replace with actual tag suffix if needed
        }
      }),
    );
    debugPrint('Response from aws bedrock: ${response.body}');
    debugPrint('Response from aws bedrock: ${response.statusCode}');

    if (response.statusCode != 200) {
      throw HttpRequestException(
        message: 'Authentication failed: ${response.reasonPhrase}',
        statusCode: response.statusCode,
      );
    }

    return response;
  }

  Map<String, String> _getCommonHeaders() {
    return {
      AWSHeaders.contentType: 'application/json', // Set content type as JSON
      AWSHeaders.accept: '*/*',
      AWSHeaders.authorization:
          'Bearer ${authCredentials.sessionTokens.idToken}', // Use the ID token for authorization
      AWSHeaders.securityToken: authCredentials.sessionTokens.sessionToken ??
          '', // Use the session token
      'X-Amzn-Bedrock-GuardrailIdentifier':
          guardRailIdentifier, // Custom header value for guardrail identifier
      'X-Amzn-Bedrock-GuardrailVersion': '1', // Version for the guardrail
      'X-Amzn-Bedrock-Trace': 'DISABLED', //Valid Values: ENABLED | DISABLED
    };
  }

ADD:

Also tried below and same error "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."

I did also cross checked with https://github.com/aws-amplify/amplify-flutter/issues/4506 to ensure signer is same and it does match for signer. Request format is as per format: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html

final awsRequest = common.AWSHttpRequest(
  method: common.AWSHttpMethod.post,
  uri: Uri.https(
    'bedrock-runtime.us-east-1.amazonaws.com',
    '/model/amazon.titan-text-express-v1/invoke',
  ),
  headers: _getCommonHeaders(),
  body: json.encode({
    'amazon-bedrock-guardrailConfig': {
      'tagSuffix': 'string',
    },
  }).codeUnits,
);

And I also tried a totally different endpoint such as userpool below:

 // THis is merely used for testing of signer
  Future<void> signAndSendRequestToUserPool() async {
    // Create the signing scope
    final scope = AWSCredentialScope(
      region: region,
      service: AWSService.cognitoIdentityProvider,
    );
    // Create AWS credentials
    final credentials = AWSCredentialsProvider(AWSCredentials(
      authCredentials.sessionTokens.accessKeyId,
      authCredentials.sessionTokens.secretAccessKey,
      authCredentials.sessionTokens.sessionToken,
      authCredentials.sessionTokens.expiration,
    ));

    // Initialize the signer
    signer = AWSSigV4Signer(
      credentialsProvider: credentials,
      algorithm: AWSAlgorithm.hmacSha256,
    );

    // Create the HTTP request
    final request = AWSHttpRequest(
      method: AWSHttpMethod.post,
      uri: Uri.https('cognito-idp.$region.amazonaws.com', '/'),
      headers: const {
        AWSHeaders.target: 'AWSCognitoIdentityProviderService.DescribeUserPool',
        AWSHeaders.contentType: 'application/x-amz-json-1.1',
      },
      body: json.encode({
        'UserPoolId': userPoolId,
      }).codeUnits,
    );

    // Sign and send the HTTP request
    final signedRequest = await signer.sign(
      request,
      credentialScope: scope,
    );

    final resp = signedRequest.send();
    final respBody = await resp.response;
    safePrint(respBody);
    debugPrint('Response from AWS Userpool: ${respBody.statusCode}');

    debugPrint(
        'Response from AWS Userpool: ${await processResponse(respBody.body)}');
  }

And for this I got different error as below. 


flutter: Response from AWS Userpool: 400
flutter: Response from AWS Userpool: {"__type":"AccessDeniedException","Message":"User: arn:aws:sts::******* is not authorized to perform: cognito-idp:DescribeUserPool on resource: arn:aws:cognito-idp:us-east-1:**************** because no identity-based policy allows the cognito-idp:DescribeUserPool action"}

What seems like, the signer seemed to have worked when used for userpool).

So my guess is signer for bedrock seems to have problem for me for reason I don't know or not sure if I am doing something wrong.

NikaHsn commented 3 weeks ago

@dkliss thank you for providing these details. we will look into this issue and get back to you with any updates.

Equartey commented 2 weeks ago

Hi @dkliss,

I was able to make a request successfully with the following snippet. The key part that got it working was reusing the S3ServiceConfiguration class when signing the request. I suspect this has something to do with the payload encoding, but need to verify that still. Once we isolate whats required for bedrock, we can expose that as its own class.

In the meantime, my example below should unblock you. Please let us know if you have any other issues.

Dart Bedrock Anthropic Example ```dart // Make a signed request to an Anthropic model on AWS Bedrock. // Signed by the the AWS SigV4 signer for Dart. // Note: Request can take several seconds to complete. Future _sendBedrockRequest() async { try { const region = 'YOUR_AWS_REGION'; const modelId = 'anthropic.claude-3-sonnet-20240229-v1:0'; // Fetch Current Auth Session final cognitoPlugin = Amplify.Auth.getPlugin(AmplifyAuthCognito.pluginKey); final session = await cognitoPlugin.fetchAuthSession(); final credentials = session.credentialsResult.value; // Create AWS credentials final credentialsProvider = AWSCredentialsProvider(credentials); // Initialize the signer final signer = AWSSigV4Signer( credentialsProvider: credentialsProvider, ); final scope = AWSCredentialScope( region: region, service: const AWSService('bedrock'), ); // Anthropic Example final payload = { "system": "You are Claude, an AI assistant created by Anthropic to be helpful, harmless, and honest. Your goal is to provide informative and substantive responses to queries while avoiding potential harms.", "anthropic_version": "bedrock-2023-05-31", "max_tokens": 1024, "messages": [ {"role": "user", "content": "Hello there."}, { "role": "assistant", "content": "Hi, I'm Claude. How can I help you?" }, {"role": "user", "content": "Can you explain LLMs in plain English?"} ] }; // Create the AWSBaseHttpRequest final awsRequest = AWSHttpRequest( method: AWSHttpMethod.post, uri: Uri.https( _bedrockEndpoint, // The hostname '/model/$modelId/invoke', // The path ), headers: const { AWSHeaders.contentType: 'application/json', AWSHeaders.accept: 'application/json', }, body: jsonEncode(payload).codeUnits, ); // Sign the request final signedRequest = await signer.sign( awsRequest, credentialScope: scope, serviceConfiguration: S3ServiceConfiguration(), ); // Send the request using AWSHttpClient final operation = signedRequest.send(); // Handle the response final response = await operation.response; final body = await _decodeResponse(response); debugPrint('Response from AWS Bedrock - Status: ${body}'); } catch (e) { debugPrint('Request failed: $e'); } } Future _decodeResponse(AWSBaseHttpResponse response) async { final sb = StringBuffer() ..writeln(response.statusCode) ..write(await utf8.decodeStream(response.split())); return sb.toString(); } ```
dkliss commented 2 weeks ago

Thanks a lot @Equartey for your response.

I can confirm I have received status code below after I change the serviceConfiguration: S3ServiceConfiguration() (I guess we will get bedRock specific serviceConfiguration in future?).

Response from AWS Bedrock - Status: 200

In case it helps, initially, even after adding your proposed serviceConfiguration, I did get response but it also had error 403 because of IAM issues as mentioned here https://repost.aws/knowledge-center/bedrock-invokemodel-api-error.

Error: "AccessDeniedException: An error occurred (AccessDeniedException) when calling the InvokeModel operation: User: <> is not authorized to perform: bedrock:InvokeModel on resource: <> because no identity-based policy allows the bedrock:InvokeModel action.

This was resolved by adding below statement in the role, which was auto created initially by auth (in sandbox, via typescript auth)

{
  "Version": "2012-10-17",
  "Statement": {
    "Sid": "AllowInference",
    "Effect": "Allow",
    "Action": [
      "bedrock:InvokeModel",
      "bedrock:InvokeModelWithResponseStream"
    ],
    "Resource": "arn:aws:bedrock:*::foundation-model/model-id"
  }

}

I couldn't find a way to update the policy via code and therefore needed to manually add this to AWS after identifying the role created by amplify sandbox? Not sure if there is any way I can adjust IAM policy within code, for example, in below auth setup. 


import { defineAuth, secret } from '@aws-amplify/backend';

export const auth = defineAuth({
  loginWith: {
    email: true,

I did see https://docs.amplify.aws/gen1/flutter/tools/cli/project/permissions-boundary/ but it gave me an error (even when I have setup a sandbox env already).

amplify env update
🛑 No Amplify backend project files detected within this folder.

Resolution: 
Either initialize a new Amplify project or pull an existing project.
- "amplify init" to initialize a new Amplify project
- "amplify pull <app-id>" to pull your existing Amplify project. Find the <app-id> in the AWS Console or Amplify Studio.
Jordan-Nelson commented 2 weeks ago

@dkliss - You should be able to modify Amplify generated resources with the AWS CDK. Here is a simple example of overriding the password policy. If you have more questions about overriding specific resources I would suggest opening an issue here: https://github.com/aws-amplify/amplify-backend.

I am going to update this to a feature request to expose the appropriate service configuration so that the workaround of using S3 config is not required.

dkliss commented 2 weeks ago

Thanks @Jordan-Nelson. I will have a look at these references. Thanks for looking into this.

GregoryPardini commented 2 weeks ago

Hi everyone, I'm experiencing the same issue described here using the provided example from the repository. Here's the code I'm using:

final host = '$bucketName.s3.$region.amazonaws.com';
final path = '/$objectKey';

final scope = AWSCredentialScope(
  region: region,
  service: AWSService.s3,
);

final signer = AWSSigV4Signer(
  credentialsProvider:
      AWSCredentialsProvider(AWSCredentials(accessKey, secretKey)),
);
final serviceConfiguration = S3ServiceConfiguration();

// Creating a pre-signed URL for downloading the file
final urlRequest = AWSHttpRequest.get(
  Uri.https(host, path),
  headers: {
    AWSHeaders.host: host,
    AWSHeaders.date: AWSDateTime.now().toString(),
    AWSHeaders.contentSHA256: 'UNSIGNED-PAYLOAD',
    // AWSHeaders.contentSHA256:
    //     hex.encode(sha256.convert(utf8.encode('')).bytes),
    // AWSHeaders.expires: expirationInSeconds.toString(),
  },
);
final signedUrl = signer.presignSync(
  urlRequest,
  credentialScope: scope,
  serviceConfiguration: serviceConfiguration,
  expiresIn: Duration(seconds: expirationInSeconds),
);

In my case, the problem occurs with the pre-signed URL, which doesn't work as expected. I followed the example exactly, but I keep receiving the SignatureDoesNotMatch error when trying to access the URL through a browser. I’ve tried verifying the contentSHA256 and other parameters, but without success. However, I've noticed that if I sign the URL and send it using the .send().response method instead of through the browser, it works correctly. This leads me to think that the issue might be related to how the browser handles the pre-signed URL or some detail in the HTTP request. Does anyone have any suggestions on what might be causing this difference in behavior? I'm happy to provide more details if needed. Thanks!

Jordan-Nelson commented 2 weeks ago

@GregoryPardini Using S3ServiceConfiguration was a workaround since there is no service configuration at the moment for bedrock and the configurations appear to be similar for the two services. If you are finding certain scenarios where this is not working, I would guess that using S3ServiceConfiguration is not a valid work around in all scenarios.