aws-cloudformation / cfn-lint

CloudFormation Linter
MIT No Attribution
2.45k stars 594 forks source link

seeing E3031 for valid SSM #3640

Closed badrobit closed 2 months ago

badrobit commented 2 months ago

CloudFormation Lint Version

1.11.0

What operating system are you using?

Mac/Ubuntu

Describe the bug

The linter is raising a E3031 error saying

'<redacted>' does not match '^(arn:(aws[a-zA-Z-]*)?:lambda:)?([a-z]{2}((-gov)|(-iso([a-z]?)))?-[a-z]+-\\d{1}:)?(\\d{12}:)?(function:)?([a-zA-Z0-9-_]+)(:(\\$LATEST|[a-zA-Z0-9-_]+))?$'

Instead of passing in the full ARN for the resource we are referencing an SSM parameter: !Ref .... That SSM parameter key <redacted> points to the value of the actual ARN.

We don't expect that string to pass that regex, because that string isn't an ARN but it's still a valid CF template.

Expected behavior

The expectation would be for it to be able to see that it is a SSM parameter key and check what that points to instead.

Reproduction template

I am not allowed to share this.

badrobit commented 2 months ago

I am told it is similar in nature to: https://github.com/aws-cloudformation/cfn-lint/issues/1570

kddejong commented 2 months ago

@badrobit what resource and property are you getting this error in?

badrobit commented 2 months ago

@badrobit what resource and property are you getting this error in?

The location where the defined string is located is:

  AuthorizerFunctionArn:
    Type: AWS::SSM::Parameter::Value<String>
    Default: <string>

The line in our template.yaml that the error is referencing is:

Resources:
kddejong commented 2 months ago

Resources? Missing something here. If I can at least get the resource type and the property I should be able to replicate the issue. I've tried a few options and cannot replicate this.

badrobit commented 2 months ago

sorry clearly not the devops person working on getting your a minimum reproducible error. Appreciate your prompt responses!

kddejong commented 2 months ago

I used this template

Parameters:
  AuthorizerFunctionArn:
    Type: AWS::SSM::Parameter::Value<String>
    Default: /prod/AuthorizerFunctionArn
Resources:
  Parameter:
    Type: AWS::Lambda::Permission
    Properties:
      Action: lambda:InvokeFunction
      FunctionName: !Ref AuthorizerFunctionArn
      Principal: apigateway.amazonaws.com
      SourceArn: !Ref AuthorizerFunctionArn

SourceArn definition

"SourceArn": {
   "maxLength": 1024,
   "minLength": 12,
   "pattern": "^arn:(aws[a-zA-Z0-9-]*):([a-zA-Z0-9\\-])+:([a-z]{2}((-gov)|(-iso([a-z]?)))?-[a-z]+-\\d{1})?:(\\d{12})?:(.*)$",
   "type": "string"
  }

With this template I don't get the error. This regex matches the regex you have above so I have to be close. So in general this should be covered but maybe they are using it in a few nested functions or maybe I'm missing something else.

badrobit commented 2 months ago

We are able to reproduce using the following template:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: mvp

Parameters:
  AuthorizerFunctionArn:
    Type: AWS::SSM::Parameter::Value<String>
    Default: authorizer.arn
    Description: Gatekeeper Authorizer Function Arn

Resources:
  RestApi:
    Type: AWS::Serverless::Api
    Properties:
      Name: my-api
      StageName: v1
      EndpointConfiguration:
        Type: REGIONAL
      BinaryMediaTypes:
        - "*~1*"
      Auth:
        Authorizers:
          GatekeeperAuth:
            FunctionArn: !Ref AuthorizerFunctionArn
        DefaultAuthorizer: GatekeeperAuth
badrobit commented 2 months ago

If we use the same template with Default: authorizer it works and we thought it might be the arn but Default authorizer.func also fails so maybe it is the "." ?

badrobit commented 2 months ago

The value you provide in your example Default: /prod/AuthorizerFunctionArn also doesn't match the regex from the error we see but the linter isn't warning on that:

image
badrobit commented 2 months ago

In case it is a bit vague this is the error we are getting when running v1.11.1 against the provided failing example

E3031 'authorizer.arn' does not match '^(arn:(aws[a-zA-Z-]*)?:lambda:)?([a-z]{2}((-gov)|(-iso([a-z]?)))?-[a-z]+-\\d{1}:)?(\\d{12}:)?(function:)?([a-zA-Z0-9-_]+)(:(\\$LATEST|[a-zA-Z0-9-_]+))?$'
template-error.yaml:11:1
kddejong commented 2 months ago

The part I was missing was the SAM transform. Which when cfn-lint run the transform the resulting template has the following.

"RestApiGatekeeperAuthAuthorizerPermission": {
      "Properties": {
       "Action": "lambda:InvokeFunction",
       "FunctionName": "authorizer.arn",

And the FunctionName is failing. I need to look into what we are passing here to SAM transform and or a workaround for this scenario.

kddejong commented 2 months ago

So to the SAM translator library we are passing the template. So in this case it looks like the SAM translator is basically subbing it in. My guess is when this operates on the service side that this value must be changed by the time the transform runs.

kddejong commented 2 months ago

@badrobit I created a related issue on the SAM translator side. I may get some push back from it but going to see if they can give me options to not have this Default value substitution happen. This would be an issue for all of our validator keywords: enum, pattern, minLength, maxLength, minimum, maximum, etc.

If I can't get a fix in SAM we will have to come up with a workaround. Off the top of my head I would probably look at template Default values of SSM parameters then in the validation of those keywords if any of the values match those SSM Default values we skip validation.

badrobit commented 2 months ago

Thanks for the super prompt responses and digging into this for us!

kddejong commented 2 months ago

I may have to implement a workaround as I have yet to hear anything on my issue.

badrobit commented 1 month ago

@kddejong what release can this be found in? I am running 1.15.1 and it is still reporting the error.

kddejong commented 1 month ago

Should be fixed now.