Circular dependency? with serverless-dynamodb-autoscaling

[cdichiara]

So, I'm hesitant to call this an "issue", because it involves the interaction with another plugin ... but I do think there's something in here that this plugin might be doing that impacts this

 The CloudFormation template is invalid: Circular dependency between resources: [irhbackendDynamoDBAutoscaleRoleSa1b7044f5b423b64d61102be5d8c83c5, irhbackendDynamoDBAutoscaleRoleN4ab25b4fbf29412c02dd379d82952dd2, irhbackendDynamoDBAutoscaleRoleUsersDbUsersemailHayesUseast1, AutoScalingNestedStack, irhbackendDynamoDBAutoscaleRoleSbba38cfb55c552f9b350c918be2291b6, irhbackendDynamoDBAutoscaleRoleD4f2163618ad4c1b80636c6d627ecc261, irhbackendDynamoDBAutoscaleRoleEventsDbEventsactionHayesUseast1, irhbackendDynamoDBAutoscaleRoleEcff19f2ee2d2c7e5b48b4d8d1d61b33b, irhbackendDynamoDBAutoscaleRoleD2d8ed570c2612e39555a6291e51ddb5b, irhbackendDynamoDBAutoscaleRoleAf856b8abc3b1b56bfcf11910f71e64a0]

If you look at the pre- and post-files for this issue: Nested Stacks.zip Resource Limit.zip

I think what you're doing is that you replace the dependencies that a role like irhbackendDynamoDBAutoscaleRoleSbba38cfb55c552f9b350c918be2291b6 has upon the scaling targets and policies that have been placed in the AutoScalingNestedStack with a dependency upon the AutoScalingNestedStack itself. On the other hand, when you declare the AutoScalingNestedStack resource down below, it has dependencies back to irhbackendDynamoDBAutoscaleRoleSbba38cfb55c552f9b350c918be2291b6 because you are passing its ARN as a parameter.

I believe this problem has been created by the dynamodb-autoscaling plugin. I believe he has created non-necessary vertical dependencies in order to avoid rate-exceeded errors. I believe he should be creating non-necessary horizontal dependencies to create a similar slow-down, which would avoid this problem here.

But you might be able to help me with the next problem ... when I move the AutoScaling roles over into the NestedStack in stacks-map.js to avoid this circular dependency:

stacksMap['AWS::IAM::Role'] = {
  destination: (resourceName) => {
    if (/AutoscaleRole/.test(resourceName)) return 'AutoScaling';

    return null;
  },
};

I get the following error: Error Message.txt

Unfortunately I don't have any config files to show you, they appear to not have been created properly. Can you see anything in my change to stacks-map.js that might cause this problem?

[cdichiara]

Also, when I use the following stacks-map.js:

stacks-map.txt

Where all of the AutoScaling is left in the main stack, the problem doesn't occur and the upload to S3 happens... it makes me think your plugin is having difficulty with something about the option where AutoScaling roles are being placed with AutoScaling policies and targets...

[dougmoscrop]

I'm fine with calling this an issue, yeah! I think either we have to not move resources like you did or we need to move more resources.

[cdichiara]

You know, when I wrote what I wrote above, I probably was thinking about how this makes sense to me... sure doesn't now. Since you're willing to put in the effort to try to figure out if anything needs fixing, let me take the time to understand it again. :smile

OK, first off, I think the first approach, which I'm describing to you above, involves a configuration like this:

stacksMap['AWS::ApplicationAutoScaling::ScalingPolicy'] = { destination: 'AutoScaling' };
stacksMap['AWS::ApplicationAutoScaling::ScalableTarget'] = { destination: 'AutoScaling' };

And the failed solution I attempt later also includes something like this:

stacksMap['AWS::IAM::Role'] = {
  destination: (resourceName) => {
    if (/AutoscaleRole/.test(resourceName)) return 'AutoScaling';

    return null;
  },
};

When I look at the pre-split-stacks solution, Resource.Limit.zip, I can see that the autoscaling plugin does things like this:

          "irhbackendDynamoDBAutoscaleRoleSbba38cfb55c552f9b350c918be2291b6": {
            "DependsOn": [
              "SponsorsDb",
              "irhbackendDynamoDBAutoscaleRoleSponsorsDbHayesUseast1",
              "irhbackendTableScalingPolicyReadSponsorsDbHayesUseast1",
              "irhbackendAutoScalingTargetReadSponsorsDbHayesUseast1",
              "irhbackendTableScalingPolicyWriteSponsorsDbHayesUseast1",
              "irhbackendAutoScalingTargetWriteSponsorsDbHayesUseast1"
            ],

And I don't think these are true dependencies, but the kinds of dependencies to slow down a deployment so that it doesn't fail due to too many concurrent creations. And I think the problem is that it causes a circular dependency between the Update and the AutoScaling stacks when someone does the approach I did above. So, perhaps the answer to approach #1 is, for the autoscaling plugin, you can't split the roles from the policies & targets because of the arbitrary dependencies that have been put there for now. I suspect it may be around here in his code.

As for approach #2, where I attempt to move the roles over to join the policies & targets, makes sense based on the problems I was having before. But the issue still appears to be something circular ... just happening before any files are generated. I thought you might know something about why this might happen, or be able to reproduce this with more ability to debug.

It'd be really cool if you were able to help folks with the autoscaling task ... I'm assuming it's commonly used and a common source of many resources. Unfortunately I think that plugin has slowed development right now.