Non-zero refresh on aws_wafv2_web_acl

t0yv0 commented 5 months ago

What happened?

This seems to be a representative issue with non-zero refresh with some trivial changes that is inherited from upstream but is highlighted more prominently for the Pulumi CLI. I am curious if our strategy would be to accept this behavior or try to suppress similar benign refresh diffs in some way.

resource "aws_wafv2_web_acl" "example" {
  name = "managed-rule-example"

  default_action {
    block {}
  }

  scope = "REGIONAL"

  rule {
    name     = "test-aws-rules"

    override_action {
      count {}
    }

    priority = 1

    statement {
      managed_rule_group_statement {
        name        = "AWSManagedRulesCommonRuleSet"
        vendor_name = "AWS"
        version = "Version_1.3"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = false
      metric_name                = "test-aws-rules"
      sampled_requests_enabled   = true
    }
  }

  visibility_config {
    cloudwatch_metrics_enabled = false
    metric_name = "josh-test"
    sampled_requests_enabled = true
  }
}

$ terraform apply # success
$ terraform state show aws_wafv2_web_acl.example

# aws_wafv2_web_acl.example:
resource "aws_wafv2_web_acl" "example" {
    arn        = "arn:aws:wafv2:us-east-1:616138583583:regional/webacl/managed-rule-example/4c058453-d339-4e59-9396-4f41b0a3e076"
    capacity   = 700
    id         = "4c058453-d339-4e59-9396-4f41b0a3e076"
    lock_token = "c1f5c390-e08f-49fb-8134-03e0760d9f93"
    name       = "managed-rule-example"
    scope      = "REGIONAL"
    tags_all   = {}

    default_action {
        block {
        }
    }

    rule {
        name     = "test-aws-rules"
        priority = 1

        override_action {
            count {}
        }

        statement {
            managed_rule_group_statement {
                name        = "AWSManagedRulesCommonRuleSet"
                vendor_name = "AWS"
                version     = "Version_1.3"
            }
        }

        visibility_config {
            cloudwatch_metrics_enabled = false
            metric_name                = "test-aws-rules"
            sampled_requests_enabled   = true
        }
    }

    visibility_config {
        cloudwatch_metrics_enabled = false
        metric_name                = "josh-test"
        sampled_requests_enabled   = true
    }
}

$ terraform refresh
aws_wafv2_web_acl.example: Refreshing state... [id=4c058453-d339-4e59-9396-4f41b0a3e076]

$ terraform state show aws_wafv2_web_acl.example
# aws_wafv2_web_acl.example:
resource "aws_wafv2_web_acl" "example" {
    arn           = "arn:aws:wafv2:us-east-1:616138583583:regional/webacl/managed-rule-example/4c058453-d339-4e59-9396-4f41b0a3e076"
    capacity      = 700
    id            = "4c058453-d339-4e59-9396-4f41b0a3e076"
    lock_token    = "c1f5c390-e08f-49fb-8134-03e0760d9f93"
    name          = "managed-rule-example"
    scope         = "REGIONAL"
    tags          = {}
    tags_all      = {}
    token_domains = []

    default_action {
        block {
        }
    }

    rule {
        name     = "test-aws-rules"
        priority = 1

        override_action {
            count {}
        }

        statement {
            managed_rule_group_statement {
                name        = "AWSManagedRulesCommonRuleSet"
                vendor_name = "AWS"
                version     = "Version_1.3"
            }
        }

        visibility_config {
            cloudwatch_metrics_enabled = false
            metric_name                = "test-aws-rules"
            sampled_requests_enabled   = true
        }
    }

    visibility_config {
        cloudwatch_metrics_enabled = false
        metric_name                = "josh-test"
        sampled_requests_enabled   = true
    }
}

Note that tags and token_domains changed from missing to an empty value, but the state was, in fact, updated. In TF CLI this seems a low-ceremony operation.

In contrast, when using Pulumi CLI with the latest bridge, we do get the exact same behavior but Pulumi CI prompts the user to confirm the refresh update. It shows tags and tokenDomains changing.

Example

See above.

Output of `pulumi about`

CLI          
Version      3.99.0
Go Version   go1.21.5
Go Compiler  gc

Plugins
NAME    VERSION
aws     6.19.0
nodejs  unknown

Host     
OS       darwin
Version  14.2.1
Arch     x86_64

This project is written in nodejs: executable='/Users/t0yv0/bin/node' version='v18.18.2'

Current Stack: t0yv0/regress-1423/repro-1423

TYPE                     URN
pulumi:pulumi:Stack      urn:pulumi:repro-1423::regress-1423::pulumi:pulumi:Stack::regress-1423-repro-1423
pulumi:providers:aws     urn:pulumi:repro-1423::regress-1423::pulumi:providers:aws::prov
aws:wafv2/webAcl:WebAcl  urn:pulumi:repro-1423::regress-1423::aws:wafv2/webAcl:WebAcl::web-acl

Found no pending operations associated with repro-1423

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/t0yv0
User           t0yv0
Organizations  t0yv0, pulumi
Token type     personal

Dependencies:
NAME            VERSION
@pulumi/aws     6.19.0
@pulumi/pulumi  3.103.1
@types/aws-sdk  2.7.0
@types/node     8.10.66

Pulumi locates its logs in /var/folders/gk/cchgxh512m72f_dmkcc3d09h0000gp/T/com.apple.shortcuts.mac-helper// by default

Additional context

N/A

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

mikhailshilkov commented 2 months ago

Do you have the corresponding Pulumi code handy?

lukehoban commented 2 months ago

In practice, this is a non-issue in Terraform. After the initial deployment of the resource - here's what each of the following shows:

$ terraform plan
aws_wafv2_web_acl.example: Refreshing state... [id=87eed556-3035-460e-815d-0156d0deba0d]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are
needed.

$ terraform apply
aws_wafv2_web_acl.example: Refreshing state... [id=87eed556-3035-460e-815d-0156d0deba0d]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Notably - after that 2nd terraform apply the refresh has run (implicitly) and the state is "good", so there is no more drift/diff.

So in practice, any Terraform user will almost never see any issue here.

In Pulumi though, since the refresh isn't run automatically, and the refresh shows a diff explicitly, users will "see" that diff when they first do an explicit preview.

The one big "difference" in display between Pulumi and Terraform is that a pulumi up --refresh in Pulumi will show the refresh diff here, but a terraform plan/terraform apply will not show the refresh diff. They will both actually make the changes associated with the refresh, Terraform just won't show them to the user, so this won't feel like a problem.

t0yv0 commented 2 months ago

Sure @mikhailshilkov the Pulumi program is as follows:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const example = new aws.wafv2.WebAcl("example", {
    name: "managed-rule-example",
    defaultAction: {block: {}},
    scope: "REGIONAL",
    rules: [
        {
            name: "test-aws-rules",
            overrideAction: {count: {}},
            priority: 1,
            statement: {
                managedRuleGroupStatement: {
                    name: "AWSManagedRulesCommonRuleSet",
                    vendorName: "AWS",
                    version: "Version_1.3",
                }
            },
            visibilityConfig: {
                cloudwatchMetricsEnabled: false,
                metricName: "test-aws-rules",
                sampledRequestsEnabled: true,
            }
        },
    ],
    visibilityConfig: {
        cloudwatchMetricsEnabled: false,
        metricName: "josh-test",
        sampledRequestsEnabled: true,
    }
})

export const exampleId = example.id;

It appears that pulumi refresh shows a plan that updates WebAcl but pulumi up --refresh doesn't, it shows WebAcl as unchanging but when pressed for details it show the same diff:

   $ pulumi refresh

   details
   pulumi:pulumi:Stack: (same)
     [urn=urn:pulumi:dev::aws-3361::pulumi:pulumi:Stack::aws-3361-dev]
     ~ aws:wafv2/webAcl:WebAcl: (update)
         [id=eddb06ed-2f55-467b-80b8-20880e017c7a]
         [urn=urn:pulumi:dev::aws-3361::aws:wafv2/webAcl:WebAcl::example]
         [provider=urn:pulumi:dev::aws-3361::pulumi:providers:aws::default_6_32_0::95c9e9b5-57d8-473b-b497-6f98f0e5f8bf]
         --outputs:--
       + tags                     : {}
       + tagsAll                  : {}
       + tokenDomains             : []

   $ pulumi up --refresh

   Previewing update (dev)

   View in Browser (Ctrl+O): https://app.pulumi.com/anton-pulumi-corp/aws-3361/dev/previews/206f4835-d9ee-4d2f-93df-eb1b23002008

      Type                 Name          Plan
      pulumi:pulumi:Stack  aws-3361-dev
      └─ aws:wafv2:WebAcl  example

   Resources:
     2 unchanged

  # Although 2 unchanged, the details still show this:

  ~ pulumi:pulumi:Stack: (refresh)
     [urn=urn:pulumi:dev::aws-3361::pulumi:pulumi:Stack::aws-3361-dev]
     ~ aws:wafv2/webAcl:WebAcl: (refresh)
         [id=eddb06ed-2f55-467b-80b8-20880e017c7a]
         [urn=urn:pulumi:dev::aws-3361::aws:wafv2/webAcl:WebAcl::example]
         [provider=urn:pulumi:dev::aws-3361::pulumi:providers:aws::default_6_32_0::95c9e9b5-57d8-473b-b497-6f98f0e5f8bf]
         --outputs:--
       + tags                     : {}
       + tagsAll                  : {}
       + tokenDomains             : []

t0yv0 commented 2 months ago

After either pulumi refresh or pulumi up --refresh is accepted, the state gets in a good place and works as expected after that.

pulumi / pulumi-aws