constant diff from wafv2 with no changes

[cbcmg]

The following wafv2 config...

  portal_frontend_whitelist_ipset = aws.wafv2.IpSet(
    f'portal-frontend-whitelist-{environment_name}',
    opts=ResourceOptions(provider=aws_provider),
    addresses=portal_frontend_whitelist,
    ip_address_version='IPV4',
    scope='REGIONAL',
    tags={
      'ResourceGroup': aws_resource_group_name
    }
  )

  portal_frontend_acl = aws.wafv2.WebAcl(
    f'portal-frontend-{environment_name}',
    opts=ResourceOptions(provider=aws_provider),
    scope='REGIONAL',
    default_action={
      'Block': {}
    },
    rules=[
      WebAclRuleArgs(
        name='AWS-AWSManagedRulesCommonRuleSet',
        priority=1,
        statement=WebAclRuleStatementArgs(
          managed_rule_group_statement=WebAclRuleStatementManagedRuleGroupStatementArgs(
            vendor_name='AWS',
            name='AWSManagedRulesCommonRuleSet',
          )
        ),
        override_action=WebAclRuleOverrideActionArgs(
          none=WebAclRuleOverrideActionNoneArgs()
        ),
        visibility_config=WebAclRuleVisibilityConfigArgs(
          sampled_requests_enabled=True,
          cloudwatch_metrics_enabled=True,
          metric_name='AWS-AWSManagedRulesCommonRuleSet',
        )
      ),

      WebAclRuleArgs(
        name='portal_frontend_whitelist_ipset',
        priority=2,
        statement=WebAclRuleStatementArgs(
          ip_set_reference_statement=WebAclRuleStatementIpSetReferenceStatementArgs(
            arn=portal_frontend_whitelist_ipset.arn,
          )
        ),
        action=WebAclRuleActionArgs(
          allow=WebAclRuleActionAllowArgs()
        ),
        visibility_config=WebAclRuleVisibilityConfigArgs(
          sampled_requests_enabled=True,
          cloudwatch_metrics_enabled=True,
          metric_name='portal_frontend_whitelist_ipset',
        )
      ),
    ],
    visibility_config={
      'cloudwatchMetricsEnabled': True,
      'metric_name': 'portal-frontend',
      'sampledRequestsEnabled': True,
    },
    tags={
      'ResourceGroup': aws_resource_group_name
    },
  )

produces this diff every time I run pulumi up. The update succeeds, with no changes as far as I can see.

  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:portal.test1::portal::pulumi:pulumi:Stack::portal-portal.test1]
    > pulumi:pulumi:StackReference: (read)
        [id=acme/xxx/xxx.devx]
        [urn=urn:pulumi:portal.test1::portal::pulumi:pulumi:StackReference::acme/xxx/xxx.devx]
        name: "acme/xxx/xxx.devx"
    ~ aws:wafv2/webAcl:WebAcl: (update)
        [id=xxx]
        [urn=urn:pulumi:portal.test1::portal::aws:wafv2/webAcl:WebAcl::portal-frontend-test1]
        [provider=urn:pulumi:portal.test1::portal::pulumi:providers:aws::aws_provider::xxxxx]
      ~ rules: [
          ~ [0]: {
                  + name            : "AWS-AWSManagedRulesCommonRuleSet"
                  + priority        : 1
                  ~ statement       : {
                      ~ managedRuleGroupStatement: {
                          + name      : "AWSManagedRulesCommonRuleSet"
                          + vendorName: "AWS"
                        }
                    }
                  ~ visibilityConfig: {
                      + cloudwatchMetricsEnabled: true
                      + metricName              : "AWS-AWSManagedRulesCommonRuleSet"
                      + sampledRequestsEnabled  : true
                    }
                }
          ~ [1]: {
                  + name            : "portal_frontend_whitelist_ipset"
                  + priority        : 2
                  ~ statement       : {
                      ~ ipSetReferenceStatement: {
                          + arn: "arn:aws:wafv2:ap-southeast-2:111111111:regional/ipset/portal-frontend-whitelist-test1-072a323/xxxxxxxxx"
                        }
                    }
                  ~ visibilityConfig: {
                      + cloudwatchMetricsEnabled: true
                      + metricName              : "portal_frontend_whitelist_ipset"
                      + sampledRequestsEnabled  : true
                    }
                }
        ]

requirements.txt

Package           Version
----------------- ---------
Arpeggio          1.10.1
attrs             20.3.0
certifi           2020.11.8
chardet           3.0.4
dill              0.3.3
grpcio            1.33.2
idna              2.10
parver            0.3.1
pip               20.2.3
protobuf          3.14.0
pulumi            2.23.2
pulumi-aws        3.35.0
pulumi-kubernetes 2.8.4
pulumi-postgresql 2.8.1
pulumi-random     3.1.1
PyYAML            5.3.1
requests          2.25.0
semver            2.13.0
setuptools        49.2.1
six               1.15.0
urllib3           1.26.2

[jre21]

Is anyone from Pulumi looking into this? We're running into the same issues with the TypeScript sdk (@pulumi/aws 4.10.0): webAcl's diff always renders as if creating all the rules from scratch, even when there are no changes to apply. #1098 identified the same problem (among others) nearly a year ago, and the lack of activity on both issues leaves me worried that this behavior will continue to fall through the cracks indefinitely.

[joeduffy]

I took a brief look and I can't figure out why this is happening. It seems to be some combination of: the way we check/diff old state versus new state, populate the inputs/outputs based on prior program runs, and/or, possibly, set diffing.

Here is some raw data. I think someone like @pgavlin, @mikhailshilkov, or @EvanBoyle will need to weigh in, however.

The rules property is the culprit. Running the program the first time yields these inputs and outputs for the rules.

Inputs:

"rules": [
    {
        "__defaults": [],
        "name": "AWS-AWSManagedRulesCommonRuleSet",
        "overrideAction": {
            "__defaults": [],
            "none": {
                "__defaults": []
            }
        },
        "priority": 1,
        "statement": {
            "__defaults": [],
            "managedRuleGroupStatement": {
                "__defaults": [],
                "name": "AWSManagedRulesCommonRuleSet",
                "vendorName": "AWS"
            }
        },
        "visibilityConfig": {
            "__defaults": [],
            "cloudwatchMetricsEnabled": true,
            "metricName": "AWS-AWSManagedRulesCommonRuleSet",
            "sampledRequestsEnabled": true
        }
    },
    {
        "__defaults": [],
        "action": {
            "__defaults": [],
            "allow": {
                "__defaults": []
            }
        },
        "name": "portal_frontend_whitelist_ipset",
        "priority": 2,
        "statement": {
            "__defaults": [],
            "ipSetReferenceStatement": {
                "__defaults": [],
                "arn": "arn:aws:wafv2:us-east-1:153052954103:regional/ipset/portal-frontend-whitelist-test-42db0ef/ecb38af0-0560-4f06-8e4c-2dc50072c66b"
            }
        },
        "visibilityConfig": {
            "__defaults": [],
            "cloudwatchMetricsEnabled": true,
            "metricName": "portal_frontend_whitelist_ipset",
            "sampledRequestsEnabled": true
        }
    }
],

Outputs:

"rules": [
    {
        "action": null,
        "name": "AWS-AWSManagedRulesCommonRuleSet",
        "overrideAction": {
            "count": null,
            "none": null
        },
        "priority": 1,
        "statement": {
            "andStatement": null,
            "byteMatchStatement": null,
            "geoMatchStatement": null,
            "ipSetReferenceStatement": null,
            "managedRuleGroupStatement": {
                "excludedRules": [],
                "name": "AWSManagedRulesCommonRuleSet",
                "scopeDownStatement": null,
                "vendorName": "AWS"
            },
            "notStatement": null,
            "orStatement": null,
            "rateBasedStatement": null,
            "regexPatternSetReferenceStatement": null,
            "ruleGroupReferenceStatement": null,
            "sizeConstraintStatement": null,
            "sqliMatchStatement": null,
            "xssMatchStatement": null
        },
        "visibilityConfig": {
            "cloudwatchMetricsEnabled": true,
            "metricName": "AWS-AWSManagedRulesCommonRuleSet",
            "sampledRequestsEnabled": true
        }
    },
    {
        "action": {
            "allow": null,
            "block": null,
            "count": null
        },
        "name": "portal_frontend_whitelist_ipset",
        "overrideAction": null,
        "priority": 2,
        "statement": {
            "andStatement": null,
            "byteMatchStatement": null,
            "geoMatchStatement": null,
            "ipSetReferenceStatement": {
                "arn": "arn:aws:wafv2:us-east-1:153052954103:regional/ipset/portal-frontend-whitelist-test-42db0ef/ecb38af0-0560-4f06-8e4c-2dc50072c66b",
                "ipSetForwardedIpConfig": null
            },
            "managedRuleGroupStatement": null,
            "notStatement": null,
            "orStatement": null,
            "rateBasedStatement": null,
            "regexPatternSetReferenceStatement": null,
            "ruleGroupReferenceStatement": null,
            "sizeConstraintStatement": null,
            "sqliMatchStatement": null,
            "xssMatchStatement": null
        },
        "visibilityConfig": {
            "cloudwatchMetricsEnabled": true,
            "metricName": "portal_frontend_whitelist_ipset",
            "sampledRequestsEnabled": true
        }
    }
],

Here is what we send over RPC during the Diff command, which perceives a difference:

Olds: 434 I0814 12:57:18.079817 12171 rpc.go:72] Marshaling property for RPC[Provider[aws, 0xc00146c2a0].Diff(urn:pulumi:dev::wafwaf::aws:wafv2/webAcl:WebAcl::portal-frontend-test,e94302a1-b7ee-4765-ba7b-35f6ce2c9a23).olds]: rules ={[{map[action:{<nil>} name:{AWS-AWSManagedRulesCommonRuleSet} overrideAction:{map[count:{<nil>} none:{<nil>}]} priority:{1} statement:{map[andStatement:{<nil>} byteMatchStatement:{<nil>} geoMatchStatement:{<nil>} ipSetRef erenceStatement:{<nil>} managedRuleGroupStatement:{map[excludedRules:{[]} name:{AWSManagedRulesCommonRuleSet} scopeDownStatement:{<nil>} vendorName:{AWS}]} notStatement:{<nil>} orStatement:{<nil>} rateBasedStatement:{<nil> } regexPatternSetReferenceStatement:{<nil>} ruleGroupReferenceStatement:{<nil>} sizeConstraintStatement:{<nil>} sqliMatchStatement:{<nil>} xssMatchStatement:{<nil>}]} visibilityConfig:{map[cloudwatchMetricsEnabled:{true} m etricName:{AWS-AWSManagedRulesCommonRuleSet} sampledRequestsEnabled:{true}]}]} {map[action:{map[allow:{<nil>} block:{<nil>} count:{<nil>}]} name:{portal_frontend_whitelist_ipset} overrideAction:{<nil>} priority:{2} stateme nt:{map[andStatement:{<nil>} byteMatchStatement:{<nil>} geoMatchStatement:{<nil>} ipSetReferenceStatement:{map[arn:{arn:aws:wafv2:us-east-1:153052954103:regional/ipset/portal-frontend-whitelist-test-42db0ef/ecb38af0-0560-4 f06-8e4c-2dc50072c66b} ipSetForwardedIpConfig:{<nil>}]} managedRuleGroupStatement:{<nil>} notStatement:{<nil>} orStatement:{<nil>} rateBasedStatement:{<nil>} regexPatternSetReferenceStatement:{<nil>} ruleGroupReferenceStat ement:{<nil>} sizeConstraintStatement:{<nil>} sqliMatchStatement:{<nil>} xssMatchStatement:{<nil>}]} visibilityConfig:{map[cloudwatchMetricsEnabled:{true} metricName:{portal_frontend_whitelist_ipset} sampledRequestsEnabled :{true}]}]}]}

News: 505 I0814 12:57:18.080419 12171 rpc.go:72] Marshaling property for RPC[Provider[aws, 0xc00146c2a0].Diff(urn:pulumi:dev::wafwaf::aws:wafv2/webAcl:WebAcl::portal-frontend-test,e94302a1-b7ee-4765-ba7b-35f6ce2c9a23).news]: rules ={[{map[__defaults:{[]} name:{AWS-AWSManagedRulesCommonRuleSet} overrideAction:{map[__defaults:{[]} none:{map[__defaults:{[]}]}]} priority:{1} statement:{map[__defaults:{[]} managedRuleGroupStatement:{map[__defaults:{[]} n ame:{AWSManagedRulesCommonRuleSet} vendorName:{AWS}]}]} visibilityConfig:{map[__defaults:{[]} cloudwatchMetricsEnabled:{true} metricName:{AWS-AWSManagedRulesCommonRuleSet} sampledRequestsEnabled:{true}]}]} {map[__defaults: {[]} action:{map[__defaults:{[]} allow:{map[__defaults:{[]}]}]} name:{portal_frontend_whitelist_ipset} priority:{2} statement:{map[__defaults:{[]} ipSetReferenceStatement:{map[__defaults:{[]} arn:{arn:aws:wafv2:us-east-1:1 53052954103:regional/ipset/portal-frontend-whitelist-test-42db0ef/ecb38af0-0560-4f06-8e4c-2dc50072c66b}]}]} visibilityConfig:{map[__defaults:{[]} cloudwatchMetricsEnabled:{true} metricName:{portal_frontend_whitelist_ipset} sampledRequestsEnabled:{true}]}]}]}

And here is the final determination resulting from that invocation of Diff, which decides there is a difference: 550 I0814 12:57:18.361747 12171 provider_plugin.go:677] Provider[aws, 0xc00146c2a0].Diff(urn:pulumi:dev::wafwaf::aws:wafv2/webAcl:WebAcl::portal-frontend-test,e94302a1-b7ee-4765-ba7b-35f6ce2c9a23) success: changes=2 #replace s=[] #stables=[name scope] delbefrepl=false, diffs=#[rules rules rules rules rules rules rules rules rules rules rules rules rules], detaileddiff=map[rules[0].name: rules[0].priority: rules[0].statement.managedRuleGroupSta tement.name: rules[0].statement.managedRuleGroupStatement.vendorName: rules[0].visibilityConfig.cloudwatchMetricsEnabled: rules[0].visibilityConfig.metricName: rules[0].visibilityConfig.sampledRequestsEnabled: rules[1].nam e: rules[1].priority: rules[1].statement.ipSetReferenceStatement.arn: rules[1].visibilityConfig.cloudwatchMetricsEnabled: rules[1].visibilityConfig.metricName: rules[1].visibilityConfig.sampledRequestsEnabled:] I0814 12:57:18.361834 12171 step_generator.go:652] Planner decided to update 'urn:pulumi:dev::wafwaf::aws:wafv2/webAcl:WebAcl::portal-frontend-test'

[bohlander]

Just hit this myself. Any workaround besides just updating each time?

[ghost]

Running into this myself. Any news on this?

[SharpEdgeMarshall]

Any news on this?

[wimoMisterX]

Anyone has a workaround for this?

[pavelescurazvan]

Same here, any news?

[nathan-knight]

I was encountering this and switched to the AWS Native provider (just for wafv2, the rest of my stack is still using the AWS Classic provider), adjusted a few parameter names (different capitalization on a few things), and it doesn't have this issue.

[Freakazoid182]

I'm also running into this issue and also tried the AWS native provider for this resource. That one has it's own issues that I didn't manage to workaround, which lead me to try this classic version.

[SharpEdgeMarshall]

This is really a pain for us, running pulumi with the CI we have all this noise in the preview output each time.

[blampe]

I can confirm we're also seeing this internally at Pulumi. I tested 3 versions of pulumi-aws (ts) which all show the bug -- 4.14, 4.28, and 5.9.2.

This is really a pain for us, running pulumi with the CI we have all this noise in the preview output each time.

Sorry you're running into this. It's the kind of bug that degrades the usefulness of the CI integration, so it's top of mind for me. If we always show the user "hey check this out!" when in reality nothing is changing, it's less likely they'll notice or care when something actually does change.

[SharpEdgeMarshall]

@blampe I would like to help solving this issue, did you already know what could be? Is the state that is not saved correctly?

[blampe]

@blampe I would like to help solving this issue, did you already know what could be? Is the state that is not saved correctly?

I'm not sure where the problem is. Last week I confirmed pulumi preview --json showed identical newState and oldState for the waf resource, but then it also included this which seems to be why it's getting replaced?

"diffReasons": [
    "rules",
    "rules",
    "rules",
    "rules",
    "rules",
    "rules",
    "rules",
    "rules",
    "rules",
    "rules",
    "rules",
    "rules",
    "rules"
],
"detailedDiff": {
    "rules[0].name": {
        "kind": "add",
        "inputDiff": false
    },

[jkodroff]

Verified that this is still an issue and also does not seem to occur with an equivalent TF template. Going to check into the Diff behavior in the bridge.

[pgavlin]

Verified that this is still an issue and also does not seem to occur with an equivalent TF template

Can you paste the TF config here for posterity?

[jkodroff]

@pgavlin Any commented out stuff is from me and @stack72 attempting to debug, but it should be pretty close to the original:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

# Configure the AWS Provider
provider "aws" {
  region = "us-east-1"
}

resource "aws_wafv2_web_acl" "web_acl" {
  name = "josh-test"

  default_action {
    block {}
  }

  scope = "REGIONAL"

  visibility_config {
    cloudwatch_metrics_enabled = false
    metric_name                = "josh-test"
    sampled_requests_enabled   = true
  }

  rule {
    name = "josh-test-aws-rules"

    override_action {
      count {}
    }

    priority = 1

    statement {
      managed_rule_group_statement {
        vendor_name = "AWS"
        name        = "AWSManagedRulesCommonRuleSet"
        # version     = "Version_1.3"
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = false
      metric_name                = "josh-test-aws-rules"
      sampled_requests_enabled   = true
    }
  }

  tags = {
    "key" = "value"
  }
}

And the equivalent Pulumi:

import * as aws from "@pulumi/aws";

new aws.wafv2.WebAcl("web-acl", {
  defaultAction: {
    block: {},
  },
  scope: "REGIONAL",
  visibilityConfig: {
    cloudwatchMetricsEnabled: false,
    metricName: "josh-test",
    sampledRequestsEnabled: true,
  },
  rules: [{
    name: "josh-test-aws-rules",
    overrideAction: {
      count: {},
    },
    priority: 1,
    statement: {
      managedRuleGroupStatement: {
        vendorName: "AWS",
        name: "AWSManagedRulesCommonRuleSet",
        // version: "Version_1.3",
      },
    },
    visibilityConfig: {
      cloudwatchMetricsEnabled: false,
      metricName: "josh-test-aws-rules",
      sampledRequestsEnabled: true,
    }
  }]
});

@stack72 and I took a look at this for several hours today. We checked the TF provider schema, nothing doing.

Here's some dumped olds and news, if it's any help:

Q LOG

[15:56:21 tfbridge/provider.go:768 github.com/pulumi/pulumi-terraform-bridge/v3/pkg/tfbridge.(*Provider).Diff]
0.001s olds=resource.PropertyMap{
           "arn": {
               V:  "arn:aws:wafv2:us-east-1:616138583583:regional/webacl/web-acl-1f4eae1/9d218587-551e-4529-95a6-32796d38a4fd",
           },
           "capacity": {
               V:  float64(700),
           },
           "customResponseBodies": {
               V:  []resource.PropertyValue{
               },
           },
           "defaultAction": {
               V:  resource.PropertyMap{
               },
           },
           "description": {},
           "id":          {
               V:  "9d218587-551e-4529-95a6-32796d38a4fd",
           },
           "lockToken": {
               V:  "61cac8fb-c9d1-41d4-bafe-63974353a5b7",
           },
           "name": {
               V:  "web-acl-1f4eae1",
           },
           "rules": {
               V:  []resource.PropertyValue{
                   {
                       V:  resource.PropertyMap{
                           "name": {
                               V:  "josh-test-aws-rules",
                           },
                           "overrideAction": {
                               V:  resource.PropertyMap{
                               },
                           },
                           "priority": {
                               V:  float64(1),
                           },
                           "ruleLabels": {
                               V:  []resource.PropertyValue{
                               },
                           },
                           "statement": {
                               V:  resource.PropertyMap{
                                   "managedRuleGroupStatement": {
                                       V:  resource.PropertyMap{
                                           "excludedRules": {
                                               V:  []resource.PropertyValue{
                                               },
                                           },
                                           "name": {
                                               V:  "AWSManagedRulesCommonRuleSet",
                                           },
                                           "vendorName": {
                                               V:  "AWS",
                                           },
                                           "version": {},
                                       },
                                   },
                               },
                           },
                           "visibilityConfig": {
                               V:  resource.PropertyMap{
                                   "cloudwatchMetricsEnabled": {},
                                   "metricName":               {
                                       V:  "josh-test-aws-rules",
                                   },
                                   "sampledRequestsEnabled": {
                                       V:  bool(true),
                                   },
                               },
                           },
                       },
                   },
               },
           },
           "scope": {
               V:  "REGIONAL",
           },
           "tags": {
               V:  resource.PropertyMap{
               },
           },
           "tagsAll": {
               V:  resource.PropertyMap{
               },
           },
           "visibilityConfig": {
               V:  resource.PropertyMap{
                   "cloudwatchMetricsEnabled": {},
                   "metricName":               {
                       V:  "josh-test",
                   },
                   "sampledRequestsEnabled": {
                       V:  bool(true),
                   },
               },
           },
       }
       news=resource.PropertyMap{
           "__defaults": {
               V:  []resource.PropertyValue{
                   {
                       V:  "name",
                   },
               },
           },
           "defaultAction": {
               V:  resource.PropertyMap{
                   "__defaults": {
                       V:  []resource.PropertyValue{
                       },
                   },
                   "block": {
                       V:  resource.PropertyMap{
                           "__defaults": {
                               V:  []resource.PropertyValue{
                               },
                           },
                       },
                   },
               },
           },
           "name": {
               V:  "web-acl-1f4eae1",
           },
           "rules": {
               V:  []resource.PropertyValue{
                   {
                       V:  resource.PropertyMap{
                           "__defaults": {
                               V:  []resource.PropertyValue{
                               },
                           },
                           "name": {
                               V:  "josh-test-aws-rules",
                           },
                           "overrideAction": {
                               V:  resource.PropertyMap{
                                   "__defaults": {
                                       V:  []resource.PropertyValue{
                                       },
                                   },
                                   "count": {
                                       V:  resource.PropertyMap{
                                           "__defaults": {
                                               V:  []resource.PropertyValue{
                                               },
                                           },
                                       },
                                   },
                               },
                           },
                           "priority": {
                               V:  float64(1),
                           },
                           "statement": {
                               V:  resource.PropertyMap{
                                   "__defaults": {
                                       V:  []resource.PropertyValue{
                                       },
                                   },
                                   "managedRuleGroupStatement": {
                                       V:  resource.PropertyMap{
                                           "__defaults": {
                                               V:  []resource.PropertyValue{
                                               },
                                           },
                                           "name": {
                                               V:  "AWSManagedRulesCommonRuleSet",
                                           },
                                           "vendorName": {
                                               V:  "AWS",
                                           },
                                       },
                                   },
                               },
                           },
                           "visibilityConfig": {
                               V:  resource.PropertyMap{
                                   "__defaults": {
                                       V:  []resource.PropertyValue{
                                       },
                                   },
                                   "cloudwatchMetricsEnabled": {},
                                   "metricName":               {
                                       V:  "josh-test-aws-rules",
                                   },
                                   "sampledRequestsEnabled": {
                                       V:  bool(true),
                                   },
                               },
                           },
                       },
                   },
               },
           },
           "scope": {
               V:  "REGIONAL",
           },
           "visibilityConfig": {
               V:  resource.PropertyMap{
                   "__defaults": {
                       V:  []resource.PropertyValue{
                       },
                   },
                   "cloudwatchMetricsEnabled": {},
                   "metricName":               {
                       V:  "josh-test",
                   },
                   "sampledRequestsEnabled": {
                       V:  bool(true),
                   },
               },
           },
       }

Per @stack72, it looks like the program itself may not be correctly translating to state, but the TF schema does not appear to have anything ususual as far as we can tell.

[igrayson]

We've worked around this with:

// NOTE: while updating "rules", you must comment out "ignoreChanges" setting.
// see https://github.com/pulumi/pulumi-aws/issues/1423
new aws.wafv2.WebAcl("xxx", {
  rules: [ ... ],
...
  }, {
    ignoreChanges: ["rules"],
  });

This is a bad workaround. rules is the main content of the declaration, so developers need to be aware of the hack when making legit changes.

[kmosher]

In playing around with a test stack, this seems to in part be related to empty actions in rules. For instance action: { block {} } or override_action: { count: {} }

For instance, this rule won't converge

        {
            action: {
                block: {},
            },
            name: "rate-limit",
            priority: 2,
            statement: {
                rateBasedStatement: {
                    limit: 30000,
                    aggregateKeyType: "IP",
                },
            },
            visibilityConfig: {
                cloudwatchMetricsEnabled: true,
                metricName: "rate-limit",
                sampledRequestsEnabled: true,
            },
        },

But changing block {} to block: { customResponse: { responseCode: 403 }}} will cause it stop emitting diffs.

Unfortunately some actions are designed to only ever be empty, (e.g. override_action: { count: {} }} ), so this is not a full solution for all rule types. That might give some hints as to where the problem lies though.

For my own purposes, I switched to the aws-native provider for this resource because it also has better behavior wrt the lockToken. Though, that version of the resource is not without it's own bugs.

[Alberto-Minehub]

any update on this one?

[will3942]

Do you have an update on this one please - or any ideas where I could start looking to contribute?

[guitarrapc]

In my case, Pulumi state WAFv2 rule index change everytime pulumi applyed. Indeed it happens nothing but annoying me identify is it real change or not.

Here's sample state diff on Pulumi. rules[0] changed to index 2, and rules[2] changes to index 0.

~  aws:wafv2/webAcl:WebAcl (update)
    [id=xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx]
    [urn=urn:pulumi:project::foo::foo:components:waf$aws:wafv2/webAcl:WebAcl::foo-waf-alb-webacl]
    rules               : [
        [0]: {
            name            : "foo-waf-rule-0" => "foo-waf-rule-2"
            priority        : 0 => 2
            statement       : {
                ipSetReferenceStatement          : {"arn":"arn:aws:wafv2:us-east-1:xxxxxxxxxxxxxx:regional/ipset/foo-waf-block-ipset/xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx","ipSetForwardedIpConfig":null} => null
            }
            visibilityConfig: {
                metricName              : "foo-waf-rule-0" => "foo-waf-rule-2"
            }
        }
        [2]: {
            name            : "foo-waf-rule-2" => "foo-waf-rule-0"
            priority        : 2 => 0
            statement       : {
                ipSetReferenceStatement          : null => {"arn":"arn:aws:wafv2:us-east-1:xxxxxxxxxxxxxx:regional/ipset/foo-waf-block-ipset/xxxxxxxx-xxxx-xxxx-xxxxxxxxxxxx","ipSetForwardedIpConfig":null}
            }
            visibilityConfig: {
                metricName              : "foo-waf-rule-2" => "foo-waf-rule-0"
            }
        }
    ]

Environment

Language: .NET
Dependencies:
NAME               VERSION
Pulumi             3.51.2
Pulumi.Aws         5.27.0

[tscully49]

Any updates on this bug? Running into the same issue with typescript.

[fancywriter]

Having exactly the same issue for a long time. Any update on this?

[fancywriter]

Could anyone give a hint where in the code this diff is actually compared so I can try to research and investigate why it happens? I see clearly from AWS CLI output that no changes need to be made... but pulumi is stubborn.

[joeduffy]

@kpitzen I see this was marked "blocked," but it wasn't clear---what is it blocked on? Is it the ongoing TF bridge work and, if so, is there enough done now that a fix is within reach? Also saw this and I wasn't sure how excited to get.

[kpitzen]

@kpitzen I see this was marked "blocked," but it wasn't clear---what is it blocked on? Is it the ongoing TF bridge work and, if so, is there enough done now that a fix is within reach? Also saw this and I wasn't sure how excited to get.

That's correct, @joeduffy - I marked it as such since this is a cross-team effort. I'm happy to remove the blocked label if we feel it's not appropriate. I don't think there's been much progress on that specific PR in a little while, as there's been a more focused effort on PF integrations, but I'd love to help move this issue forward however I can!

[AaronFriel]

@t0yv0 is this a good candidate to trial through the Plugin Framework upgrade & see if the diff error here reproduces when the resource uses the PF bridge?

[t0yv0]

Yes. https://github.com/pulumi/pulumi-terraform-bridge/pull/887 pins down the exact root cause and has a fix for the issue and is updated with the discussion of tradeoffs. I'm concerned deploying it 0-100% everywhere but we could deploy #887, possibly flagged, so it only applies to say this one resource.

One alternative we discussed is finding a way to forward-port this one resource to Plugin Framework ahead of upstream migrating it to PF, which automatically moves us to the latest and more straightforward diff implementation. Unlike 887 though, we haven't done the work yet to prove out though that it's possible and solves the issue.

[iwahbe]

https://github.com/pulumi/pulumi-aws/pull/2604 will fix this. The change will be released as part of v6.0.0.

[t0yv0]

Unfortunately our users report the issue resurfacing under a slightly different configuration on v6 in https://github.com/pulumi/pulumi-aws/issues/2664 I'm reopening for now. Reopening for now. We'll be targeting a patch release following the first v6 release to land a comprehensive fix.

[t0yv0]

We believe we have a fix for this issue that's part of the v6 release candidate and will be released as pulumi-aws v6.x.x. I will close the issue once that's released.

[t0yv0]

The fix should be available in v6.0.2, covering https://github.com/pulumi/pulumi-aws/issues/2664 and other scenarios. Please let us know if there's any remaining problems around this!

[dougludlow]

I'm still seeing this issue, using pulumi v3.106.0 and v6.22.2 of the aws plugin. Not sure if there was a regression or not? Happy to provide any other details to help get this resolved.

[larrywax]

Hi @t0yv0, would it be possible to reopen the issue? pulumi-aws v6.22.0 seems to have a regression, I'm experiencing the same behaviour as @dougludlow.

EDIT: the regression on the aws:wafv2:WebAcl resource seems to occur from v6.20.0 onwards. We have the same issue on the aws:wafv2:RuleGroup resource, we have tried going back to v6.15.0 but the diff is still there.

[iwahbe]

Let's continue this discussion in https://github.com/pulumi/pulumi-aws/issues/3454.