aws-cloudformation / cloudformation-cli

The CloudFormation Provider Development Toolkit allows you to author your own resource providers and modules that can be used by CloudFormation.
Apache License 2.0
318 stars 161 forks source link

Implement canary file generation functionality from contract test inp… #1069

Closed rajdnp closed 4 months ago

rajdnp commented 4 months ago

Description of changes: Implement canary file generation functionality from contract test inputs. The 'cfn init' command will add canarySettings to enable canary generation. Based on the canarySettings, the 'cfn generate' command will generate canary files for contract test inputs.

Testing : Scenarios :

  1. Validated if current resources are not affected by the change
  2. Validated if this is enabled by default for new resources.
  3. Validate if canary files are not generated when setting is manually turned off
  4. Validate if all the input files are generated
  5. Validate if canaries are generated, only the valid input files.
Positive case: 

"canarySettings": {
        "file_generation_enabled": false,
        "contract_test_file_names": [
            "inputs_1.json","inputs_3.json"
        ]
    }
cfn generate 

(env) ➜  testcfncli cd canary-bundle/canary 
(env) ➜  canary ls -lrt
total 16
-rw-r--r--  1 xxx  staff  513 May 21 09:48 canary1_001.yaml
-rw-r--r--  1 xxx  staff  425 May 21 09:48 canary2_001.yaml
(env) ➜  canary cat canary2_001.yaml
Description: Canary template for AWS::Route53::RecordSet
Resources:
  RecordSetCanary:
    Properties:
      GeoLocation:
        ContinentCode: NA
      HostedZoneId:
        Fn::ImportValue: awsroute53recordsetcto1
      Name: 3-8921be3b-3a99-4015-a8c3-ae7545aacd7a.ct.test
      ResourceRecords:
      - 192.0.2.98
      SetIdentifier: geolocationrecordset
      TTL: '900'
      Type: A
    Type: AWS::Route53::RecordSet
(env) ➜  canary cd ../                 
(env) ➜  canary-bundle ls     
bootstrap.yaml canary

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

joshuadeanhall commented 4 months ago

Should canary generation be part of the main CLI or be a new plugin?

rajdnp commented 4 months ago

Should canary generation be part of the main CLI or be a new plugin? Idea is to generate canary files for 3P resources as well. As there is no canary framework for 3p, Service team can use these files manually if required. It is a two-way door. If we see issues, we can easily restrict the file generation only to AWS/AMZN prefix.