Default model needs special access

[ssolkhon]

Hey,

I am trying to spin up this example but the default model needs you to request access via AWS and unless you're an enterprise it's unlikely that access will be granted. As this is an example it would make much more sense to use a base model available to anyone who has added the model access to their account so that the barrier to entry is low.

When using the default configuration you will get the following error on a standard AWS account:

2024-06-03T16:12:09.876Z    ea1bf956-215e-4a01-abae-e89fd3f5f144    INFO    ValidationException: You don't have access to the model with the specified model ID. (Service: BedrockRuntime, Status Code: 403, Request ID: 30a65748-7fad-41bf-af5a-9e0dd6b0dbe7)
    at de_ValidationExceptionRes (/var/runtime/node_modules/@aws-sdk/client-bedrock-agent-runtime/dist-cjs/index.js:1128:21)
    at de_CommandError (/var/runtime/node_modules/@aws-sdk/client-bedrock-agent-runtime/dist-cjs/index.js:1004:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
    at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/core/dist-cjs/index.js:165:18
    at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
    at async /var/runtime/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22
    at async /var/task/index.js:420:22
    at async runRequest (/var/task/index.js:151:26) {
  '$fault': 'client',
  '$metadata': {
    httpStatusCode: 400,
    requestId: 'f097936d-cbe7-4dc2-a401-cf4234b7a654',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  }
}

If you then try to change the base model to something else you'll get a different error telling you that the base model is not supported:

2024-06-03T16:21:09.813Z    f265beec-0674-4ab8-99fd-038a53afce59    INFO    ValidationException: The model arn provided is not supported. Please check your configuration and retry the request.
    at de_ValidationExceptionRes (/var/runtime/node_modules/@aws-sdk/client-bedrock-agent-runtime/dist-cjs/index.js:1128:21)
    at de_CommandError (/var/runtime/node_modules/@aws-sdk/client-bedrock-agent-runtime/dist-cjs/index.js:1004:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-serde/dist-cjs/index.js:35:20
    at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/core/dist-cjs/index.js:165:18
    at async /var/runtime/node_modules/@aws-sdk/node_modules/@smithy/middleware-retry/dist-cjs/index.js:320:38
    at async /var/runtime/node_modules/@aws-sdk/middleware-logger/dist-cjs/index.js:33:22
    at async /var/task/index.js:420:22
    at async runRequest (/var/task/index.js:151:26) {
  '$fault': 'client',
  '$metadata': {
    httpStatusCode: 400,
    requestId: '15694efd-47dd-47e7-9a0f-7841c5b6a5a4',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  }
}

I tried meta.llama3-8b-instruct-v1:0 and amazon.titan-text-lite-v1 so maybe I've just been unlucky with picking the wrong models to use instead. If you could point me in the right direction of a model that will work or let me know where I have gone wrong that would be greatly appreciated! To change the base model I updated the file backend/lambda/query/index.js with the above models.

Thanks.

[ssolkhon]

Edit: I found an alternative model that works - amazon.titan-text-premier-v1:0.

[ssolkhon]

Reopening to see if there is appetite from a core reviewer to change the default model.

[akomandooru]

Thank you for your feedback - we have an item in our backlog to inject the model ID so there is flexibility to try different models without having to change code or deployment. We will share an update here after we make the change.