Bug: SNS Event does not support FilterPolicy with nested JSON objects of different depths

[bschug]

Description:

When using the SNS event type in the Events property of an AWS::Serverless::Function, we can define a FilterPolicy. SNS / CloudFormation allow us to build filters that match on nested hierarcies of JSON objects, e.g. {"status":["confirmed"], "before":{"owner":["0x0"]}}. SAM fails to create a resource with such a policy. SAM should support any FilterPolicy that is accepted by SNS and/or CloudFormation.

The issue does not appear for {"before": {"owner":["0x0"]}} or {"status": ["confirmed"]}. It only appears when mixing different depths in the same policy.

Steps to reproduce:

Try do build and deploy a stack like this:

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31

Resources:

  Topic:
    Type: AWS::SNS::Topic

  ConsumerFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/ConfirmedFunction
      Handler: ConfirmedFunction::ConfirmedFunction.Function::FunctionHandler
      Runtime: dotnet6
      Architectures:
        - arm64
      Events:
        SNSEvent:
          Type: SNS
          Properties:
            Topic: !GetAtt Topic.TopicArn
            SqsSubscription: true
            FilterPolicyScope: MessageBody
            FilterPolicy:
              status:
                - confirmed
              before:
                owner:
                  - "0x0"

Observed result:

The deployment fails with: Error: Failed to create changeset for the stack: bugreport, ex: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state: For expression "Status" we matched expected path: "FAILED" Status: FAILED. Reason: Transform AWS::Serverless-2016-10-31 failed with: Invalid Serverless Application Specification document. Number of errors found: 1. Resource with id [ConsumerFunctionSNSEvent] is invalid. Type of property 'FilterPolicy' is invalid.

Expected result:

The deployment succeeds.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

{
  "version": "1.84.0",
  "system": {
    "python": "3.8.8",
    "os": "Windows-10-10.0.19041-SP0"
  },
  "additional_dependencies": {
    "docker_engine": "24.0.2",
    "aws_cdk": "Not available",
    "terraform": "Not available"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

Note that the expected stack can be deployed by writing the subscription in plain CloudFormation:

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31

Resources:

  Topic:
    Type: AWS::SNS::Topic

  Queue:
    Type: AWS::SQS::Queue

  ConsumerFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/ConfirmedFunction
      Handler: ConfirmedFunction::ConfirmedFunction.Function::FunctionHandler
      Runtime: dotnet6
      Architectures:
        - arm64
      Events:
        EventQueue:
          Type: SQS 
          Properties:
            Queue: !GetAtt Queue.Arn

  Subscription:
    Type: AWS::SNS::Subscription 
    Properties:
      Protocol: sqs 
      RawMessageDelivery: true 
      Endpoint: !GetAtt Queue.Arn 
      TopicArn: !Ref Topic 
      FilterPolicyScope: MessageBody
      FilterPolicy:
        status:
          - confirmed
        before:
          owner:
            - "0x0"

[lucashuy]

Hi, thanks for reporting this! I can reproduce your issue where the serverless Function isn't accepting the dictionary of values (eg. {before: {owner: ["0x0"]}}) while placing leaving the policy in the normal CFN resource works.

This seems to be related to the validate of the property (expects the value to be a list of strings or objects). Moving to the SAM repo.

[xazhao]

Hi thanks for reporting this bug. This bug is related to wrong validation of FilterPolicy property. Will create a PR to fix it soon.

[GavinZZ]

@lucashuy The PR linked above is merged and should be rolling out in the upcoming weeks. Going to close this issue. Feel free to re-open or create a new issue if you have additional questions.