Open jgo80 opened 1 month ago
Hey guys, I get the same error:
The CloudFormation deployment has failed. Caused By: ❌ Deployment failed: Error [ValidationError]: Circular dependency between resources: [auth179371D7, data7552DF31, function1351588B]
Here is the code I added to backend.ts:
const SMSResponseEventSource = new DynamoEventSource(backend.data.resources.tables['SMSResponse'], {
startingPosition: StartingPosition.LATEST,
});
const EmailResponseEventSource = new DynamoEventSource(
backend.data.resources.tables['EmailResponse'],
{
startingPosition: StartingPosition.LATEST,
},
);
const VoiceCallResponseEventSource = new DynamoEventSource(
backend.data.resources.tables['VoiceCallResponse'],
{
startingPosition: StartingPosition.LATEST,
},
);
backend.OnAppointmentReminderDeliveryHandler.resources.lambda.addEventSource(
SMSResponseEventSource,
);
backend.OnAppointmentReminderDeliveryHandler.resources.lambda.addEventSource(
EmailResponseEventSource,
);
backend.OnAppointmentReminderDeliveryHandler.resources.lambda.addEventSource(
VoiceCallResponseEventSource,
);
I'm hoping this gets fixed soon so I don't have to set this all up manually.
I also get the same error when trying to set up environment variables. The weird things with this is that the error won't always happen immediately. Instead, I'll start getting the error after making other unrelated changes to my sandbox. Then I have to comment out the environment variable code in my backend.ts file to get rid of the circular dependency.
The CloudFormation deployment has failed. Caused By: ❌ Deployment failed: Error [ValidationError]: Circular dependency between resources: [auth179371D7, data7552DF31, function1351588B]
Commenting out these lines in backend.ts resolves the circular dependency:
const copyDynamoDBTableLambdaFunction = backend.CopyDynamoDBTableHandler.resources
.lambda as Function;
copyDynamoDBTableLambdaFunction.addEnvironment('APP_SYNC_API_ID', backend.data.apiId);
Assigning to @AnilMaktala to try & repro -- I'm not able to get this error to occur.
Just wanted to pop in to say that I also am getting the same error when trying to add in event source for dynamodb streams. Same issue circular dependency between auth, data, and function which doesn't make any sense to me since my lambda needs the table but my table doesn't need my lambda?
I also get the same issue as @MattWlodarski when setting up env variables.
The issue should be marked as bug rather than as question. Otherwise @AnilMaktala @palpatim proof your guide works correctly.
@jgo80 do you have a workaround by any chance for now?
@jgo80 do you have a workaround by any chance for now?
I don't 😞 Just waiting for an answer as a desperate customer moving from Gen1 to Gen2...
I found a similar issue with a resolution: https://github.com/aws-amplify/amplify-backend/issues/1552
I'm not sure if it exactly applies to our situation because I didn't quite follow the recommended solution. Let me know if it helps you guys.
Thanks @MattWlodarski I will take a look at this today and give it a try and then update this thread
Yeah the code provided in the document operates normally when only event mapping is used, but a circular dependency error occurs as soon as data custom query is used. So if you are using a custom query (both query and mutation), you have to manually make a separate cdk stack for the streaming target table and set the event mapping and policy on that stack.
As @MattWlodarski mentioned, i wrote down that issue and found the solution. the well-worked code is on my repository (you can find it on that mentioned issue), so you might be able to refer it.
I had some triggers in the auth resource. If I remove them, it removes the circular dependency.
triggers: {
customMessage,
postConfirmation,
preSignUp,
defineAuthChallenge,
createAuthChallenge,
verifyAuthChallengeResponse: verifyAuthChallenge,
},
@ggj0418 @MattWlodarski do you either of you have any auth triggers?
import { defineAuth, secret } from "@aws-amplify/backend";
import { customMessage } from "./custom-message/resource";
import { postConfirmation } from "./post-confirmation/resource";
import { preSignUp } from "./pre-signup/resource";
import { defineAuthChallenge } from "./define-auth-challenge/resource";
import { createAuthChallenge } from "./create-auth-challenge/resource";
import { verifyAuthChallenge } from "./verify-auth-challenge/resource";
/**
* Define and configure your auth resource
* @see https://docs.amplify.aws/gen2/build-a-backend/auth
*/
export const auth = defineAuth({
loginWith: {
email: true,
phone: true,
},
triggers: {
customMessage,
postConfirmation,
preSignUp,
defineAuthChallenge,
createAuthChallenge,
verifyAuthChallengeResponse: verifyAuthChallenge,
},
});
@LukaASoban hm... no i do not have any auth trigger. did you add that trigger with custom query or something else component of amplify gen2 backend?
@LukaASoban on another project, im using that auth trigger with data custom queries, but it does not have a circular error
@AnilMaktala here is the min code required for me to reproduce
/**
* @see https://docs.amplify.aws/react/build-a-backend/ to add storage, functions, and more
*/
const backend = defineBackend({
auth,
data,
dynamodbStreamPlaces,
});
const eventSource = new DynamoEventSource(
backend.data.resources.tables["Place"],
{
startingPosition: StartingPosition.LATEST,
}
);
backend.dynamodbStreamPlaces.resources.lambda.addEventSource(eventSource);
and then my auth resource.ts
/**
* Define and configure your auth resource
* @see https://docs.amplify.aws/gen2/build-a-backend/auth
*/
export const auth = defineAuth({
loginWith: {
email: true,
phone: true,
},
triggers: {
customMessage,
},
});
and my data resource.ts
import { type ClientSchema, a, defineData } from "@aws-amplify/backend";
const schema = a.schema({
Place: a
.model({
id: a.id(),
globalId: a.string().required(),
name: a.string().required(),
description: a.string(),
coordinates: a.customType({
latitude: a.float().required(),
longitude: a.float().required(),
}),
address: a.string().required(),
createdAt: a.datetime(),
updatedAt: a.datetime(),
defaultPhotoId: a.id(),
})
.authorization((allow) => [
allow.guest().to(["read"]),
allow.authenticated().to(["read", "create"]),
]),
});
export type Schema = ClientSchema<typeof schema>;
export const data = defineData({
schema,
authorizationModes: {
defaultAuthorizationMode: "userPool",
},
});
and my customMessage function
import { defineFunction } from "@aws-amplify/backend";
export const customMessage = defineFunction({
name: "custom-message",
});
import type { CustomMessageTriggerHandler } from "aws-lambda";
export const handler: CustomMessageTriggerHandler = async (event) => {
return event;
};
@LukaASoban I think that dynamoDB event mapping is the point. here is my suggestion.
const backend = defineBackend({
auth,
//preSignUp,
//storage,
data,
//getUsersByEmailOrPhoneFunction,
//createAuthChallenge,
//checkIfCognitoUserExistsFunction,
//onUpload,
dynamodbStreamPlaces,
});
const placeTable = backend.data.resources.tables["Place"]
backend.dynamodbStreamPlaces.resources.lambda.role?.attachInlinePolicy(
new Policy(
Stack.of(placeTable),
"DynamoDBPolicy",
{
statements: [
new PolicyStatement({
effect: Effect.ALLOW,
actions: [
"dynamodb:DescribeStream",
"dynamodb:GetRecords",
"dynamodb:GetShardIterator",
"dynamodb:ListStreams",
],
resources: ["*"],
}),
],
}
)
);
new EventSourceMapping(
Stack.of(placeTable),
"test",
{
target: backend.dynamodbStreamPlaces.resources.lambda,
eventSourceArn: placeTable.tableStreamArn,
startingPosition: StartingPosition.LATEST,
}
);
Oh i forgot to tell the resolved code's branch. here is the branch. it is not a main. i'm sorry for the missing description.
@ggj0418 Thanks for your input here. I seem to be getting this error when I try that:
CREATE_FAILED | AWS::Lambda::EventSourceMapping | data/amplifyData/Place/DynamodbStreamPlacesEventSourceMapping (DynamodbStreamPlacesEventSourceMappingE2426243) Resource handler returned message: "Invalid request provided: Stream arn:aws:dynamodb:us-east-2:*****:table/Place-2xgy7oir4fa5tndcygcxeusazu-NONE/stream/2024-05-30T20:36:57.557 is Disabled. You cannot create a lambda mapping on a stream that is Disabled. (Service: Lambda, Status Code: 400, Request ID: 1578ca62-c2ba-4039-ab59-15f48b13ba0e)" (RequestToken: ea51563f-a008-1cb5-0938-f9869001ab6f, HandlerErrorCode: InvalidRequest)
Not sure why my lambda stream is disabled
@LukaASoban hm... could i see your dynamdbStreamPlaces' handler?
Looks exactly like in the docs:
import type { DynamoDBStreamHandler } from "aws-lambda";
import { Logger } from "@aws-lambda-powertools/logger";
const logger = new Logger({
logLevel: "INFO",
serviceName: "dynamodb-stream-handler",
});
export const handler: DynamoDBStreamHandler = async (event) => {
for (const record of event.Records) {
logger.info(`Processing record: ${record.eventID}`);
logger.info(`Event Type: ${record.eventName}`);
if (record.eventName === "INSERT") {
// business logic to process new records
logger.info(`New Image: ${JSON.stringify(record.dynamodb?.NewImage)}`);
}
}
logger.info(`Successfully processed ${event.Records.length} records.`);
return {
batchItemFailures: [],
};
};
@ggj0418 So I deleted all my resources and re-created them in my sandbox and it worked! Thanks for the work-around.
I still get a circ dep when I add in env vars though
@LukaASoban oh yeah sandbox's hotswap is not perfectly worked... do you talk about the amplify env vars?
How come the official docs publishes a procedure that clearly causes this issue? Does no one checks what is being published?
This should be marked as an issue and fixed.
@heclon are you also getting this same error?
Hey @LukaASoban I also had an auth trigger as well. I appreciate you guys explaining the workaround but I agree with @heclon. This should be marked as an issue and fixed asap. I can't believe they haven't caught this already. They must just test their code in the simplest possible ways. It doesn't take much to trigger either of these circular dependencies...
HI @jgo80, We are able to replicate this issue and marking this as bug for the team to evaluate further.
I'm facing the same issue.
I have tried ggj0418 solution but getting errors regarding permissions. Still haven't got the work around to work. Is there any other way to get a DynamoDBStream as an event source.
This is delaying deployment of a production site.
Is there any update on the time frame for this bug to be fixed?
Before opening, please confirm:
JavaScript Framework
Not applicable
Amplify APIs
GraphQL API
Amplify Version
v6
Amplify Categories
auth, function, api
Backend
Amplify Gen 2 (Preview)
Environment information
Describe the bug
Follwing the official Gen 2 Guide to configure a Lambda function with an Amazon DynamoDB stream as an event source I get the following error during deployment/sandbox:
The error happens when I add the Event Source according to the official Docs:
Edit: I was concerned my Auth preTokenGeneration handler caused this, but it dit not. When I remove preTokenGeneration from triggers, I still get the error:
So the circular dependency definitely results from the fact that the backend resource table is referenced in the eventSource and then being injected back into the backend object. Design flaw?
Expected behavior
It should be possible to add the event source for the lambda according to the docs to invoke the lambda by DynamoDB Stream.
Reproduction steps
Just follow your own official guide: https://docs.amplify.aws/react/build-a-backend/functions/examples/dynamo-db-stream/
Code Snippet
Log output
aws-exports.js
No response
Manual configuration
No response
Additional configuration
No response
Mobile Device
No response
Mobile Operating System
No response
Mobile Browser
No response
Mobile Browser Version
No response
Additional information and screenshots
No response