template validation errors should be fatal

[rittneje]

Name of the resource

AWS::Timestream::Table

Resource Name

No response

Issue Description

Note: This seems to apply to all CloudFormation resources. I just picked AWS::Timestream::Table because that is one I know is affected.

There is an existing bug in CDK that causes it to generate invalid templates. See https://github.com/aws/aws-cdk/issues/8996

However, the bigger issue is that CloudFormation does not actually reject the template. Instead, it only spits out a warning to the event log like so:

Resource template validation failed for resource table as the template has invalid properties. Please refer to the resource documentation to fix the template. Properties validation failed for resource table with message: #/RetentionProperties: extraneous key [memoryStoreRetentionPeriodInHours] is not permitted #/RetentionProperties: extraneous key [magneticStoreRetentionPeriodInDays] is not permitted #/MagneticStoreWriteProperties: required key [EnableMagneticStoreWrites] not found #/MagneticStoreWriteProperties: extraneous key [enableMagneticStoreWrites] is not permitted #/MagneticStoreWriteProperties: extraneous key [magneticStoreRejectedDataLocation] is not permitted

Expected Behavior

CloudFormation should fail if the template contains unrecognized properties.

Observed Behavior

It logs a warning but otherwise ignores the invalid properties.

Test Cases

{
 "Resources": {
  "timestreambucket4DAC1000": {
   "Type": "AWS::S3::Bucket",
   "Properties": {
    "PublicAccessBlockConfiguration": {
     "BlockPublicAcls": true,
     "BlockPublicPolicy": true,
     "IgnorePublicAcls": true,
     "RestrictPublicBuckets": true
    }
   },
   "UpdateReplacePolicy": "Retain",
   "DeletionPolicy": "Retain",
   "Metadata": {
    "aws:cdk:path": "test-stack/timestream-bucket/Resource"
   }
  },
  "timestreambucketPolicy1FB2D719": {
   "Type": "AWS::S3::BucketPolicy",
   "Properties": {
    "Bucket": {
     "Ref": "timestreambucket4DAC1000"
    },
    "PolicyDocument": {
     "Statement": [
      {
       "Action": "s3:*",
       "Condition": {
        "Bool": {
         "aws:SecureTransport": "false"
        }
       },
       "Effect": "Deny",
       "Principal": {
        "AWS": "*"
       },
       "Resource": [
        {
         "Fn::GetAtt": [
          "timestreambucket4DAC1000",
          "Arn"
         ]
        },
        {
         "Fn::Join": [
          "",
          [
           {
            "Fn::GetAtt": [
             "timestreambucket4DAC1000",
             "Arn"
            ]
           },
           "/*"
          ]
         ]
        }
       ]
      }
     ],
     "Version": "2012-10-17"
    }
   },
   "Metadata": {
    "aws:cdk:path": "test-stack/timestream-bucket/Policy/Resource"
   }
  },
  "database": {
   "Type": "AWS::Timestream::Database",
   "Properties": {
    "DatabaseName": "MyDatabase"
   },
   "Metadata": {
    "aws:cdk:path": "test-stack/database"
   }
  },
  "table": {
   "Type": "AWS::Timestream::Table",
   "Properties": {
    "DatabaseName": "MyDatabase",
    "MagneticStoreWriteProperties": {
     "enableMagneticStoreWrites": true,
     "magneticStoreRejectedDataLocation": {
      "s3Configuration": {
       "bucketName": {
        "Ref": "timestreambucket4DAC1000"
       },
       "encryptionOption": "SSE_S3"
      }
     }
    },
    "RetentionProperties": {
     "magneticStoreRetentionPeriodInDays": "365",
     "memoryStoreRetentionPeriodInHours": "1"
    }
   },
   "Metadata": {
    "aws:cdk:path": "test-stack/table"
   }
  },
  "CDKMetadata": {
   "Type": "AWS::CDK::Metadata",
   "Properties": {
    "Analytics": "v2:deflate64:H4sIAAAAAAAA/0WMXQrCMBCEz9L3ZG0seAD1AKX1XTZxxZg0gfwgJeTuNlTwaWa+GeYIpx76Dj+Rq4fhVksoc0Jl2IbuJQ5QzlkZSuzydD+3y+itVusf77mypBeKKRAuULbyigklRmrDG0pLtTY7UfQ5KKpsXNPLu8MAQoDo3lFrHrJrLzDt+gV+FBtspAAAAA=="
   },
   "Metadata": {
    "aws:cdk:path": "test-stack/CDKMetadata/Default"
   }
  }
 },
 "Parameters": {
  "BootstrapVersion": {
   "Type": "AWS::SSM::Parameter::Value<String>",
   "Default": "/cdk-bootstrap/hnb659fds/version",
   "Description": "Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"
  }
 },
 "Rules": {
  "CheckBootstrapVersion": {
   "Assertions": [
    {
     "Assert": {
      "Fn::Not": [
       {
        "Fn::Contains": [
         [
          "1",
          "2",
          "3",
          "4",
          "5"
         ],
         {
          "Ref": "BootstrapVersion"
         }
        ]
       }
      ]
     },
     "AssertDescription": "CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."
    }
   ]
  }
 }
}

Other Details

No response

[rittneje]

Adding to the weirdness, even though CloudFormation spits out a validation error, the template is applied correctly anyway. It seems like the validation logic is case-sensitive but the apply logic isn't?

[sethusrinivasan]

@rittneje , Can you pls check if you still see this issue?

[rehanvdm]

@sethusrinivasan Yes this is still an issue

[kddejong]

With this release we should see the failures going forward:

https://aws.amazon.com/about-aws/whats-new/2024/03/aws-cloudformation-new-validation-checks-stack-operations/

from template

Resources:
  Bucket:
    Type: AWS::S3::Bucket
    Properties:
      bucketName: !Sub "${AWS::StackName}-${AWS::Region}"

That being said I'm looking into why the template you provided doesn't provide the same error.