search

aws-cloudformation / cfn-lint

CloudFormation Linter
MIT No Attribution
2.45k stars 593 forks source link

E0001 Error for SAR Apps #700

Closed jacknagz closed 5 years ago

jacknagz commented 5 years ago

cfn-lint version: cfn-lint 0.15.0

Description of issue

While running cfn-lint on a nested CloudFormation stack containing AWS::Serverless::Application resources, it errors with:

Resources:
  AppName:
    Type: AWS::Serverless::Application
    Properties:
      Location:
        ApplicationId: arn:aws:serverlessrepo:us-west-2:<my-account-id>:applications/<app-name>
        SemanticVersion: 1.0.0
      Parameters:
        Debug: 'True'
        MemorySizeMB: 128
        TimeoutSec: 300
$ cfn-lint template.yml

E0001 Resource with id [AppName] is invalid. Application with id arn:aws:serverlessrepo:us-west-2:<my-account-Id-hosting-the-app>:applications/<app-name> could not be found.
template.yml:1:1

Supporting evidence:

  1. The Application definitely exists, since the template runs with no issues.
  2. I have admin permissions with the current user to make, update, view, delete the app.
cmmeyer commented 5 years ago

Interesting. Validation of serverless resources occurs via the SAM validation libraries. Do you happen to have the SAM CLI installed? I'm curious what results you get validating through the CLI.

jacknagz commented 5 years ago

@cmmeyer I get the following:

$ sam validate --template infra.yml
infra.yml is a valid SAM Template
kddejong commented 5 years ago

@cmmeyer we actually use their Translator. https://github.com/aws-cloudformation/cfn-python-lint/blob/master/src/cfnlint/transform.py#L118 Before SAM SAR we would validate the translated templated from SAM. What I think we need to figure out is what Translator does with a SAM SAR application. My basic understanding is in the end its just a nested template. My hope is it isn't reaching out to SAM SAR to validate and pull information about the application.

jacknagz commented 5 years ago

Oddly enough, I'm having another issue with a different SAM template running in CI (no nested stacks or anything, just some AWS::Serverless resources):

E0001 Error transforming template: 'module' object has no attribute 'logging'

Really not sure where to start with this one. It's on the latest version of cfn-lint too... Any ideas @cmmeyer ?

cmmeyer commented 5 years ago

. @Dunedan figured out the logging issue: #702 So it looks like you may have gotten sam-translator 1.10.0 in the mix at some point. I wonder if the other issues may stem from that as well.

fatbasstard commented 5 years ago

Not solving it, but is it an idea to output the transformed template with --debug?

fatbasstard commented 5 years ago

Playing around with this and I think I got it.

(I'm running sam-translater 1.9.0, so it's not related to 1.10.)

Based on this:

jacknagz commented 5 years ago

Can you check the privileges on your side? I think it has to do something with that

I'm using admin privs, so it can't be that.

kddejong commented 5 years ago

I think @fatbasstard meant your AWS privileges. Since it looks like SAR is causing it to reach out to AWS you will need credentials and privileges to query it. We have tried to remain completely offline during all of our linting but this new feature in SAM has changed that. I want to dig a little farther into why its doing that, etc. To see if we can stop it or prevent it.

fatbasstard commented 5 years ago

Might even be the region.. Which region is the function in, and against what region is the linter running?

It might search for the function in another region, which causes the "cannot find it". The linter is running against us-east-1 if no region is specified

kddejong commented 5 years ago

My guess is we are going to have to translate this before we go to SAM to prevent it reaching out to the account. This is an example of what gets output on full transformation. The Template URL is probably provided by the call out to the serverless repo. The good part is we don't worry about the validity of the Url only the properties and their values for formatting reasons. What I'm proposing is we translate AWS::Serverless::Application to its equivalent resource AWS::CloudFormation::Stack voiding the TemplateURL. Or maybe there is a clever way to provide the feedback to the call that samtranslator is making to provide it a generic TemplateURL.

"Bucket": {
      "Type": "AWS::CloudFormation::Stack",
      "Properties": {
        "TemplateURL": "https://awsserverlessrepo-changesets-plntc6bfnfj.s3.amazonaws.com/AIDAJEF4MAIJQFOIKLXX6-51d20263-e5dc-461c-8d34-029afde3a90f.yaml?X-Amz-Security-Token=FQoGZXIvYXdzEAoaDLPbjmWojhcH4U1rcyKCApkuQ%2ByITP3xcIIJAQO5Ehty%2F6GWw2rHaNaxECDXgItWZh1Q6cPt0HXeTEWo8nGuohPU6IluBfzuuFfh5c6gXC3JXzygRxbloe3rSljQaI3G3ior7LqTEARPcauKlEV1uSwQ%2BS0PsiJfv8QNms0%2BV46w8iPrgDBwTrTJt%2FgLhHykWM15M4nbql1OmlC4x4866A%2FqQ5Qwtgc60Q%2FWxu4sqEj9J6y9ElwGsLUAe5oZTp%2FcnOzmkEh%2FwDfeBBq60tcZpCm0YAKHU6HB9G39Uhkze%2Bps3Y1CgTR6PHO2a71oHJJKKP88QNzIJhzTjoLXjqnSJhv8O7aaREQhqd6lVDMKBync%2FCimuZ%2FkBQ%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20190312T164245Z&X-Amz-SignedHeaders=host&X-Amz-Expires=21599&X-Amz-Credential=ASIAQ63C33MXBRRTMRDP%2F20190312%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=40f006d885d7f4b2af8da077e1ef09d9c5ecda6c4d3bab5ae288d1bb9713fb65",
        "Parameters": {
          "BucketName": "test-123456789012-us-east-1"
        },
        "Tags": [
          {
            "Value": "SAM",
            "Key": "lambda:createdBy"
          },
          {
            "Value": "arn:aws:serverlessrepo:us-east-1:123456789012:applications/test",
            "Key": "serverlessrepo:applicationId"
          },
          {
            "Value": "0.0.1",
            "Key": "serverlessrepo:semanticVersion"
          }
        ]
      }
    }
kddejong commented 5 years ago

Found a way to translate this. Release v0.17.0 is being built now and will have this fix.

cjongseok commented 5 years ago

Oddly enough, I'm having another issue with a different SAM template running in CI (no nested stacks or anything, just some AWS::Serverless resources):

E0001 Error transforming template: 'module' object has no attribute 'logging'

Really not sure where to start with this one. It's on the latest version of cfn-lint too... Any ideas @cmmeyer ?

I have the same issue as @jacknagz . I am using cfn-lint 0.22.0 for AWS:Serverless:Function as below:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31 # It transforms SAM syntax to Lambda function CloudFormation template: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/transform-aws-serverless.html
Resources:
  MyLambda:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
    ...

@kddejong Does it make sense to re-open this issue?

kddejong commented 5 years ago

Do you know what version of aws-sam-translator you have?

cjongseok commented 5 years ago

@kddejong I am using aws-sam-translator 1.12.0.

dinvlad commented 4 years ago

We still have this issue, has a fix been released yet? Thanks

kddejong commented 4 years ago

@dinvlad what version of aws-sam-translator and cfn-lint are you using? There are two issues in this one which one are you referring to?

@cjongseok are you still having this issue? We require 1.15.1 now.

dinvlad commented 4 years ago

@kddejong OK, I think I misdirected the issue. We stumbled upon this in the regular sam validate, not cfn-lint. The latter seems to work fine!