CloudFormation template is invalid on initial deploy. API Keys are not in use.

[pettyalex]

I'm seeing failure on the Validating template stage without the API Keys feature in use. I'd be glad to provide more information to get this resolved, as my team would really love to be able to use this. I've looked at the generated files in the .serverless directory, and I see the "ApiGatewayDeployment1513894811881" key is populated only in the cloudformation-template-update-stack.json. There it correctly DependsOn all of our apigateway functions, which is all identical behavior to deploying without the alias.

I can't think of many newer or unusual features we might be using other than request authorizers. So far to debug it, I've tried removing functions, and I still see the failure even with only a single function present. Hopefully something leaps out at you from my plugin list as incompatible so I have somewhere else to continue looking.

Serverless: Validating template...

  Error --------------------------------------------------

  The CloudFormation template is invalid: Template format error: Unresolved resource dependencies [ApiGatewayDeployment1513894811881] in the Resources block of the template

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Stack Trace --------------------------------------------

Error: The CloudFormation template is invalid: Template format error: Unresolved resource dependencies [ApiGatewayDeployment1513894811881] in the Resources block of the template
    at provider.request.catch (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/plugins/aws/deploy/lib/validateTemplate.js:25:13)
From previous event:
    at AwsDeploy.validateTemplate (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/plugins/aws/deploy/lib/validateTemplate.js:20:12)
From previous event:
    at AwsDeploy.BbPromise.bind.then (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/plugins/aws/deploy/index.js:117:39)
From previous event:
    at Object.aws:deploy:deploy:validateTemplate [as hook] (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/plugins/aws/deploy/index.js:113:10)
    at BbPromise.reduce (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/classes/PluginManager.js:366:55)
From previous event:
    at PluginManager.invoke (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/classes/PluginManager.js:366:22)
    at PluginManager.spawn (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/classes/PluginManager.js:384:17)
    at AwsDeploy.BbPromise.bind.then (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/plugins/aws/deploy/index.js:91:48)
From previous event:
    at Object.deploy:deploy [as hook] (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/plugins/aws/deploy/index.js:87:10)
    at BbPromise.reduce (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/classes/PluginManager.js:366:55)
From previous event:
    at PluginManager.invoke (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/classes/PluginManager.js:366:22)
    at PluginManager.run (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/classes/PluginManager.js:397:17)
    at variables.populateService.then (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/Serverless.js:104:33)
    at runCallback (timers.js:789:20)
    at tryOnImmediate (timers.js:751:5)
    at processImmediate [as _immediateCallback] (timers.js:722:5)
From previous event:
    at Serverless.run (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/lib/Serverless.js:91:74)
    at serverless.init.then (/Users/alex.petty/.nvm/versions/node/v8.9.1/lib/node_modules/serverless/bin/serverless:42:50)
    at <anonymous>

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Forums:        forum.serverless.com
     Chat:          gitter.im/serverless/serverless

  Your Environment Information -----------------------------
     OS:                     darwin
     Node Version:           8.9.1
     Serverless Version:     1.24.1

our current plugins in use are:

plugins:
    - serverless-webpack
    - serverless-domain-manager
    - serverless-kms-secrets
    - serverless-plugin-scripts
    - serverless-plugin-warmup
    - serverless-external-s3-event
    - serverless-aws-alias
    - serverless-offline # keep this plugin last

[HyperBrain]

Hi @pettyalex ,

thank you for the report. For me it looks like one of the plugins adds a resource that is not yet supported by the alias plugin. The reason for such errors is, that some of the resources are moved to the alias stack, but references to any resources that stay in the main stack must be converted by the plugin. So, if any resource that is created by a plugin, stays in the main stack, but maybe has a moved resource in DependsOn, it will break.

I'm sure that's something easy to fix in the plugin, as soon as we found what exactly it is.

The candidates, creating such unsupported resources are (the warmup plugin is fully compatible):

- serverless-domain-manager
- serverless-kms-secrets
- serverless-external-s3-event

Can you try to add only one of these 3 plugins at a time and see, which one makes it break? Afterwards, please do a serverless package with the malicious configuration and paste it here. Then I can exactly see, where the stale reference is, and add the support to the alias plugin.

[HyperBrain]

see #83: the plugin is currently not compatible with the domain manager plugin

[pettyalex]

It also seems to be incompatible with - serverless-external-s3-event. My team is investigating workarounds. I will provide you with a package once we've determined how many other plugins we can remove.

[pettyalex]

Disregard the above comment concerning serverless-external-s3-event. It seems like this may be another issue entirely. We've got many entries with null values in the DependsOn within the cloudformation-template-update-alias-stack.json after eliminating the domain manager plugin.

    "MyTeamsLambdaPermission": {
      "Type": "AWS::Lambda::Permission",
      "Properties": {
        "FunctionName": {},
        "Action": "lambda:InvokeFunction",
        "Principal": "apigateway.amazonaws.com",
        "SourceArn": {
          "Fn::Join": [
            "",
            [
              "arn:aws:execute-api:",
              {
                "Ref": "AWS::Region"
              },
              ":",
              {
                "Ref": "AWS::AccountId"
              },
              ":",
              {
                "Fn::ImportValue": "our-teams-specific-name"
              },
              "/*/*"
            ]
          ]
        }
      },
      "DependsOn": [
        null,
        null
      ]
    },

[pettyalex]

Within apiGateway.js, this is almost certainly happening because we can't find the versionName and aliasName for some reason, explaining the two nulls in the dependsOn field that don't exist in the non-alias update-stack file:

        // Adjust permission to reference the function aliases
        _.forOwn(apiLambdaPermissions, (permission, name) => {
            const functionName = _.replace(name, /LambdaPermissionApiGateway$/, '');
            const versionName = _.find(_.keys(versions), version => _.startsWith(version, functionName));
            const aliasName = _.find(_.keys(aliases), alias => _.startsWith(alias, functionName));

            // Adjust references and alias permissions
            permission.Properties.FunctionName = { Ref: aliasName };
            if (permission.Properties.SourceArn) {
                // Authorizers do not set the SourceArn property
                permission.Properties.SourceArn = {
                    'Fn::Join': [
                        '',
                        [
                            'arn:aws:execute-api:',
                            { Ref: 'AWS::Region' },
                            ':',
                            { Ref: 'AWS::AccountId' },
                            ':',
                            { 'Fn::ImportValue': `${stackName}-ApiGatewayRestApi` },
                            '/*/*'
                        ]
                    ]
                };
            }

            // Add dependency on function version
            permission.DependsOn = [ versionName, aliasName ];

            delete stageStack.Resources[name];
        });

Edit: We have no AWS::Lambda::Version, hence versions is an empty array, as well as aliases.

P.S. Thanks for all your help, as I'm digging through this I'm seeing all the comments you've left in issues in the Serverless project over the years.

[HyperBrain]

@pettyalex Thanks for the evaluation. Especially your last comment is very helpful.

Do you have the versioning feature disabled in Serverless? I remember that the default was to create versions - only if you set the flag in your serverless.yml it will be turned off. This can be at least one reason for the version resource missing issue.

Additionally it would be great, if you could check, if there are any AWS::Lambda::Version type resources in the generated CF template. The alias resources need them and depend on them as well, as an alias can only be attached to a version.

[anacronw]

I've also run into this issue. The only plugins we use are:

  - serverless-offline
  - serverless-domain-manager
  - serverless-aws-alias

And the error:

Error: The CloudFormation template is invalid: Template format error: Unresolved resource dependencies [ApiGatewayDeployment1516926540691] in the Resources block of the template