dgellow
commented
3 months ago Hi @daniel-gofore,
Could you try to run the demo example and tell me if you see the same problem?
Closed daniel-gofore closed 2 months ago
dgellow
commented
3 months ago Hi @daniel-gofore,
Could you try to run the demo example and tell me if you see the same problem?
daniel-gofore
commented
3 months ago Hi @dgellow, I have tried the demo one and I get the same result. These are the models that I have access with AWS on my account:
When I am quering the aws cli for models to see which models I have access to, I get this list : aws bedrock list-foundation-models --region=eu-west-1 --by-provider anthropic --query "modelSummaries[*].modelId" [ "anthropic.claude-3-sonnet-20240229-v1:0:28k", "anthropic.claude-3-sonnet-20240229-v1:0", "anthropic.claude-3-haiku-20240307-v1:0:48k", "anthropic.claude-3-haiku-20240307-v1:0" ]
When sending the request to Anthropic: For the models that I don't have access to I get this response: error: { message: "You don't have access to the model with the specified model ID." } For the models that I have access to but I don't use the specific format like: model: "arn:aws:bedrock:region:eu-west-1:custom-model/anthropic.claude-v2:1:200k" , I get this response: message: "1 validation error detected: Value 'anthropic.claude-3-sonnet-20240229-v1:0:28k' at 'modelId' failed to satisfy constraint: Member must satisfy regular expression pattern: (arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:(([0-9]{12}:custom-model/[a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}/[a-z0-9]{12})|(:foundation-model/[a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}([.:]?[a-z0-9-]{1,63}))|([0-9]{12}:imported-model/[a-z0-9]{12})|([0-9]{12}:provisioned-model/[a-z0-9]{12})))|([a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}([.:]?[a-z0-9-]{1,63}))|(([0-9a-zA-Z][_-]?)+)|(arn:aws(|-us-gov|-cn|-iso|-iso-b):bedrock:(|[0-9a-z-]{1,20}):(|[0-9]{12}):model-gateway/[a-zA-Z0-9-]+)" } For the models that I have access to and use the correct format, I am getting : { Output: { __type: 'com.amazon.coral.service#UnknownOperationException' }, Version: '1.0' }
dgellow
commented
3 months ago Thank you for the detail, we are looking into it
daniel-gofore
commented
3 months ago Hi, any updates on this ? Thank you.
daniel-gofore
commented
3 months ago Never mind I have fix it. The isue is that this documentation is wrong. I am using nextjs as an API server. This is the code I have used to get things working, so configure your aws cli with the correct details in order to access the your AWS account, check which AWS Anthropic services you have enabled and drop this code, make sure you have the right nameing of your model: `import { BedrockRuntimeClient, InvokeModelCommand, } from "@aws-sdk/client-bedrock-runtime"; import { NextApiRequest, NextApiResponse } from "next";
/**
@property {string} text */
/**
@property {ResponseContent[]} content */
/**
@property {completion} text */
/**
@param {string} [modelId] - The ID of the model to use. Defaults to "anthropic.claude-v2". */
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
if (req.method === 'GET') {
try {
const response = await invokeTextCompletionsApi('Hello', 'anthropic.claude-v2');
res.status(200).json(response);
}catch (error) {
console.error('Error handling request:', error);
res.status(500).json({ error: 'Failed to process request' });
}
} else {
res.setHeader('Allow', ['POST']);
res.status(405).end(Method ${req.method} Not Allowed);
}
}
/**
@param {string} [modelId] - The ID of the model to use. Defaults to "anthropic.claude-v2". */ export const invokeTextCompletionsApi = async ( prompt: string, modelId = "anthropic.claude-v2", ) => { // Create a new Bedrock Runtime client instance. const client = new BedrockRuntimeClient({ region: "eu-central-1" });
// Prepare the payload for the Text Completions API, using the required prompt template.
const enclosedPrompt = Human: ${prompt}\n\nAssistant:;
const payload = {
prompt: enclosedPrompt,
max_tokens_to_sample: 500,
temperature: 0.5,
stop_sequences: ["\n\nHuman:"],
};
// Invoke Claude with the payload and wait for the response. const command = new InvokeModelCommand({ contentType: "application/json", body: JSON.stringify(payload), modelId, }); const apiResponse = await client.send(command);
// Decode and return the response. const decoded = new TextDecoder().decode(apiResponse.body); /* @type {TextCompletionsResponseBody} / const responseBody = JSON.parse(decoded); return responseBody.completion; };`
rattrayalex
commented
2 months ago Glad you were able to figure this out!
I have this small project where I am trying to send some text to Claude. No matter what I do I can't seem to get Claude-v2:1:200k to analyze it. Here is my approach using typescript: