aws-cloudformation / cloudformation-guard

Guard offers a policy-as-code domain-specific language (DSL) to write rules and validate JSON- and YAML-formatted data such as CloudFormation Templates, K8s configurations, and Terraform JSON plans/configurations against those rules. Take this survey to provide feedback about cfn-guard: https://amazonmr.au1.qualtrics.com/jfe/form/SV_bpyzpfoYGGuuUl0
Apache License 2.0
1.3k stars 180 forks source link

[GENERAL ISSUE] cfn-guard and Terraform JSON output #518

Closed meraj-kashi closed 5 months ago

meraj-kashi commented 5 months ago

Hi! I am trying to use the cfn-guard tool to validate Terraform resources. The below sample s3 terraform resource:

provider "aws" {
  region = "us-west-2"
}

resource "aws_s3_bucket" "example" {
  bucket = "my-tf-test-bucket"
}

According to the cfn-guard native behaviour, I create a JSON output file of the Terraform resources using the commands below:

terraform plan -out=tfplan
terraform show -json tfplan > output.json

As a simple proof of concept (PoC), I am trying to validate if the AWS S3 resource contains tags or not. First, let's look at the JSON file created using the above commands:

{
    "format_version": "1.2",
    "terraform_version": "1.7.3",
    "planned_values": {
        "root_module": {
            "resources": [
                {
                    "address": "aws_s3_bucket.example",
                    "mode": "managed",
                    "type": "aws_s3_bucket",
                    "name": "example",
                    "provider_name": "registry.terraform.io/hashicorp/aws",
                    "schema_version": 0,
                    "values": {
                        "bucket": "my-tf-test-bucket",
                        "force_destroy": false,
                        "tags": null,
                        "timeouts": null
                    },
                    "sensitive_values": {
                        "cors_rule": [],
                        "grant": [],
                        "lifecycle_rule": [],
                        "logging": [],
                        "object_lock_configuration": [],
                        "replication_configuration": [],
                        "server_side_encryption_configuration": [],
                        "tags_all": {},
                        "versioning": [],
                        "website": []
                    }
                }
            ]
        }
    },
    "resource_changes": [
        {
            "address": "aws_s3_bucket.example",
            "mode": "managed",
            "type": "aws_s3_bucket",
            "name": "example",
            "provider_name": "registry.terraform.io/hashicorp/aws",
            "change": {
                "actions": [
                    "create"
                ],
                "before": null,
                "after": {
                    "bucket": "my-tf-test-bucket",
                    "force_destroy": false,
                    "tags": null,
                    "timeouts": null
                },
                "after_unknown": {
                    "acceleration_status": true,
                    "acl": true,
                    "arn": true,
                    "bucket_domain_name": true,
                    "bucket_prefix": true,
                    "bucket_regional_domain_name": true,
                    "cors_rule": true,
                    "grant": true,
                    "hosted_zone_id": true,
                    "id": true,
                    "lifecycle_rule": true,
                    "logging": true,
                    "object_lock_configuration": true,
                    "object_lock_enabled": true,
                    "policy": true,
                    "region": true,
                    "replication_configuration": true,
                    "request_payer": true,
                    "server_side_encryption_configuration": true,
                    "tags_all": true,
                    "versioning": true,
                    "website": true,
                    "website_domain": true,
                    "website_endpoint": true
                },
                "before_sensitive": false,
                "after_sensitive": {
                    "cors_rule": [],
                    "grant": [],
                    "lifecycle_rule": [],
                    "logging": [],
                    "object_lock_configuration": [],
                    "replication_configuration": [],
                    "server_side_encryption_configuration": [],
                    "tags_all": {},
                    "versioning": [],
                    "website": []
                }
            }
        }
    ],
    "configuration": {
        "provider_config": {
            "aws": {
                "name": "aws",
                "full_name": "registry.terraform.io/hashicorp/aws",
                "expressions": {
                    "region": {
                        "constant_value": "us-west-2"
                    }
                }
            }
        },
        "root_module": {
            "resources": [
                {
                    "address": "aws_s3_bucket.example",
                    "mode": "managed",
                    "type": "aws_s3_bucket",
                    "name": "example",
                    "provider_config_key": "aws",
                    "expressions": {
                        "bucket": {
                            "constant_value": "my-tf-test-bucket"
                        }
                    },
                    "schema_version": 0
                }
            ]
        }
    },
    "timestamp": "2024-06-06T12:27:13Z",
    "errored": false
}

Now, creating the below guard definition to detect the s3 resource without tags:

let s3_bucket = planned_values.root_module.resources.*[ type == "aws_s3_bucket"]

rule assert_all_resources_have_non_empty_tags when %s3_bucket !empty {
    %s3_bucket.values.tags !empty
}

And finally using cfn-guard cli to test the JSON file:

cfn-guard test --rules-file terraform_rules.guard --test-data output.json

Then getting the below error message which looks like complain about the JSON:

Error processing Parser Error when parsing `Unable to process data in file output.json, Error invalid type: map, expected a sequence at line 1 column 0.

As you can see, the JSON file has a valid format (already validated by another tool), so I cannot find the issue. I would appreciate it if you could help me identify the problem. Additionally, are there any examples of Terraform validation?

Extra details:

cfn-guard 3.1.1
Operating System: Mac
Mac version: Sonoma 14.3.1 
joshfried-aws commented 5 months ago

Hi @meraj-kashi the problem is you are using the test command. This is not the structure the test command expects. The test command is for writing unit tests for your cfn-guard rules. If you are looking to validate the template, please use the validate command.

Please let me know if there is anything else I can do to help.

joshfried-aws commented 5 months ago

Hi @meraj-kashi I am going to go ahead and close out this issue. Feel free to re-open if need be.

Thanks