Closed gpavlov2016 closed 2 months ago
Hi @gpavlov2016, thanks for raising this issue.
Copy/pasting the function from the docs example seems to work fine, but I am able to reproduce the issue using the hanlder logic you provided.
Not sure why exactly this was causing the window is not defined
error but the issue seems to be the importing of the MODEL_ID
from the resource file rather than just the Schema type, probably importing something else from that file that is using the window
object.
I basically just replaced the import of MODEL_ID
to be an environment as shown in the docs example and that seems to work.
// amplify/data/resource.ts
export const MODEL_ID = "anthropic.claude-3-haiku-20240307-v1:0";
export const generateHaikuFunction = defineFunction({
entry: "generateHaiku.ts",
environment: {
MODEL_ID,
},
});
// generate-haiku.ts
import type { Schema } from "./resource";
import {
BedrockRuntimeClient,
InvokeModelCommand,
InvokeModelCommandInput,
} from "@aws-sdk/client-bedrock-runtime";
export const handler: Schema["generateHaiku"]["functionHandler"] = async (
event,
context
) => {
// User prompt
const prompt = event.arguments.prompt;
const AWS_REGION = "us-east-1";
const messages = [
{
role: "user",
content: `${prompt}
Here is the context: ######`,
},
];
// Create a new Bedrock Runtime client instance.
const client = new BedrockRuntimeClient({ region: AWS_REGION });
// Prepare the payload for the model.
const payload = {
anthropic_version: "bedrock-2023-05-31",
max_tokens: 2000,
messages: messages,
// temperature: 0,
// top_p: 1,
};
const command = new InvokeModelCommand({
contentType: "application/json",
body: JSON.stringify(payload),
modelId: process.env.MODEL_ID,
});
const response = await client.send(command);
// Parse the response and return the generated haiku
const data = JSON.parse(Buffer.from(response.body).toString());
return data.content[0].text;
};
Coming back to this, I tried the code I shared above a few times and noticed that the query would error intermittently. I also can't see any error messages in the cloudwatch logs.
My best guess is that perhaps the requests are being rate limited but sometimes I get nothing but Unhandled lambda errors for quite a while and randomly a query will work.
I'll keep digging into this, discuss with the team, and come back with any findings.
Weirdly enough, the snippet from the docs works every time. Only have issues with the code above but there's no obvious difference that might explain what the issue is 🤔
I've transferred this ticket from amplify-js
as this might be backend related. If not, is there potentially something incorrect in the docs linked in the ticket description?
Hey @gpavlov2016 :wave: the error is a bit interesting. Is the source code shared in the original post the full handler code? The dependencies used there shouldn't produce a webpack/handlebars error, nor look for a window
global. Is that an error you're seeing in your frontend from calling the Lambda, or the error in your Lambda's execution logs?
The error is showing when amplify is trying to update the stack while you have 'npx amplify sandbox' running and saving the file. Therefore I don't think it's a backend issue, most likely related to webpack. My original handler was larger but I narrowed it down to the instantiation of the bedrock runtime module. If I remove the statement everything works. As soon as I add it and save the error is shown, before the code is even deployed
Ah interesting! Would you mind sharing the output of npx amplify info
? From @chrisbonifacio 's notes earlier, does this issue go away after removing the MODEL_ID import? https://github.com/aws-amplify/amplify-backend/issues/1589#issuecomment-2140600266
is this the piece causing issues?
// Create a new Bedrock Runtime client instance.
const client = new BedrockRuntimeClient({ region: AWS_REGION });
Closing the issue due to inactivity. Do reach out to us if you are still experiencing this issue.
Before opening, please confirm:
JavaScript Framework
Next.js
Amplify APIs
Predictions
Amplify Version
v6
Amplify Categories
function, predictions
Backend
Amplify Gen 2 (Preview)
Environment information
Describe the bug
Following the documentation here, the invocation of the function produces an error while importing the Bedrock Runtime. Function:
Error:
Expected behavior
Function should execute without errorsThe function
Reproduction steps
Observe the error in Cloudwatch for the handler and in the
errors
member above.Code Snippet
Log output
aws-exports.js
No response
Manual configuration
No response
Additional configuration
No response
Mobile Device
No response
Mobile Operating System
No response
Mobile Browser
No response
Mobile Browser Version
No response
Additional information and screenshots
No response