Unable to update awsnative.wafv2.WebAcl without updating tags.

henriiik commented 2 years ago

Hello!

Vote on this issue by adding a 👍 reaction
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already)

Issue details

When i try to make changes to the rules property of my awsnative.wafv2.WebAcl, pulumi reports that it can make an update (rather than a replace). However when the update runs i get the following error:

error: operation error CloudControl: UpdateResource, https response error StatusCode: 400, RequestID: b76eef48-8d30-4f5e-9242-5c3b98323fb2, api error ValidationException: Model validation failed (#/Tags: expected minimum item count: 1, found: 0)

I believe that this is related to the fact that the tags output of my resource is an empty array. I do have tags for my resource, but the wafv2 api does not return them, so i think that is why they are not in the outputs. (the tags are present in the inputs when looking at the stack state, but not the outputs.)

Steps to reproduce

Create a basic awsnative.wafv2.WebAcl
Run pulumi up
Update the code so the rules are changed.
Run pulumi up again
Crash

Expected: My rule should be updated and my update succeed. Actual: The api call crashes and the update fails.

here the code for a minimal reproduction (repo link):

import * as awsnative from "@pulumi/aws-native";

const webAcl = new awsnative.wafv2.WebACL(`web-acl`, {
    defaultAction: { allow: {} },
    scope: "REGIONAL",
    visibilityConfig: {
        // trigger an update every time
        metricName: `${Date.now()}`,
        cloudWatchMetricsEnabled: false,
        sampledRequestsEnabled: false
    },
    tags: [
        {
            key: "key",
            value: "val"
        }
    ]
});

here is the output of running when pulumi up to create the acl:

$ pulumi up                                                                                                                                                                                                              Previewing update (dev)

View Live: https://app.pulumi.com/henriiik/waf/dev/previews/32874831-1e44-4adb-a661-3d73454b5027

     Type                        Name     Plan       
 +   pulumi:pulumi:Stack         waf-dev  create     
 +   └─ aws-native:wafv2:WebACL  web-acl  create     

Resources:
    + 2 to create

Do you want to perform this update? yes
Updating (dev)

View Live: https://app.pulumi.com/henriiik/waf/dev/updates/6

     Type                        Name     Status      
 +   pulumi:pulumi:Stack         waf-dev  created     
 +   └─ aws-native:wafv2:WebACL  web-acl  created     

Resources:
    + 2 created

Duration: 10s

and then another one to trigger the bug:

$ pulumi up                                                                                                                                                                                                              
Previewing update (dev)

View Live: https://app.pulumi.com/henriiik/waf/dev/previews/4dac5228-b286-4ba1-9d63-1fcbbeb17993

     Type                        Name     Plan       Info
     pulumi:pulumi:Stack         waf-dev             
 ~   └─ aws-native:wafv2:WebACL  web-acl  update     [diff: ~visibilityConfig]

Resources:
    ~ 1 to update
    1 unchanged

Do you want to perform this update? yes
Updating (dev)

View Live: https://app.pulumi.com/henriiik/waf/dev/updates/7

     Type                        Name     Status                  Info
     pulumi:pulumi:Stack         waf-dev  **failed**              1 error
 ~   └─ aws-native:wafv2:WebACL  web-acl  **updating failed**     [diff: ~visibilityConfig]; 1 error

Diagnostics:
  pulumi:pulumi:Stack (waf-dev):
    error: update failed

  aws-native:wafv2:WebACL (web-acl):
    error: operation error CloudControl: UpdateResource, https response error StatusCode: 400, RequestID: 2ed1d1fd-0209-42ce-babb-0b6e80bd1121, api error ValidationException: Model validation failed (#/Description: failed validation constraint for keyword [pattern])

Resources:
    1 unchanged

Duration: 4s

guineveresaenger commented 2 years ago

Hi @henriiik - thank you for reporting this issue.

To more easily assist you, could you:

let us know which version of Pulumi and aws-native you're using?
show us a code snippet you're experiencing the issue with? it will help reproduce the bug more easily. :)

henriiik commented 2 years ago

Hello @guineveresaenger, thank you for replying!

I have updated the issue with a minimal reproduction using the latest version, 0.14.0.

guineveresaenger commented 2 years ago

Thank you so much - I see the behavior as well.

I'm going to ask @danielrbradley or @viveklak to verify whether WebAcl is fully supported in this package at this point in time.

In the meantime, I would recommend using the AWS Classic provider instead.

viveklak commented 2 years ago

@danielrbradley this looks like a bug in CloudControl - we would expect the get to CloudControl to return the tags as well if the outputs don't contain the tags.

@henriiik ~~could you try using an ignoreChanges clause on tags to get around this for now?~~

Edit: See https://github.com/pulumi/pulumi-aws-native/issues/415#issuecomment-1275609023 instead.

henriiik commented 2 years ago

I tried to run the code in the example repo i created again, but this time i got a different error. It's complaining that the description is empty. (Same as before creating the resource worked, only updates don't). So i ran a more elaborate test.

I created one resource with each of these configurations

undefined tags and undefined description
undefined tags (defined description)
undefined description (defined tags)
both defined

The creation of all resources succeeded as can be seen here:

$ pulumi up --yes
Previewing update (dev)

View Live: https://app.pulumi.com/henriiik/waf/dev/previews/ba1aa4a6-8ee8-4af1-bad8-d6993a90a44e

     Type                        Name                            Plan       
 +   pulumi:pulumi:Stack         waf-dev                         create     
 +   ├─ aws-native:wafv2:WebACL  web-acl-no-description          create     
 +   ├─ aws-native:wafv2:WebACL  web-acl-no-tags-no-description  create     
 +   ├─ aws-native:wafv2:WebACL  web-acl                         create     
 +   └─ aws-native:wafv2:WebACL  web-acl-no-tags                 create     

Resources:
    + 5 to create

Updating (dev)

View Live: https://app.pulumi.com/henriiik/waf/dev/updates/20

     Type                        Name                            Status      
 +   pulumi:pulumi:Stack         waf-dev                         created     
 +   ├─ aws-native:wafv2:WebACL  web-acl                         created     
 +   ├─ aws-native:wafv2:WebACL  web-acl-no-tags-no-description  created     
 +   ├─ aws-native:wafv2:WebACL  web-acl-no-description          created     
 +   └─ aws-native:wafv2:WebACL  web-acl-no-tags                 created     

Resources:
    + 5 created

Duration: 49s

However, on update the resources with one or more defined failed, with an error message indicating that the undefined property failed validation. But the resource with both defined was updated successfully.

$ pulumi up --yes
Previewing update (dev)

View Live: https://app.pulumi.com/henriiik/waf/dev/previews/b739427e-fd64-4663-aede-847f8ce2a929

     Type                        Name                            Plan       Info
     pulumi:pulumi:Stack         waf-dev                                    
 ~   ├─ aws-native:wafv2:WebACL  web-acl-no-tags-no-description  update     [diff: ~visibilityConfig]
 ~   ├─ aws-native:wafv2:WebACL  web-acl                         update     [diff: ~visibilityConfig]
 ~   ├─ aws-native:wafv2:WebACL  web-acl-no-description          update     [diff: ~visibilityConfig]
 ~   └─ aws-native:wafv2:WebACL  web-acl-no-tags                 update     [diff: ~visibilityConfig]

Resources:
    ~ 4 to update
    1 unchanged

Updating (dev)

View Live: https://app.pulumi.com/henriiik/waf/dev/updates/21

     Type                        Name                            Status                  Info
     pulumi:pulumi:Stack         waf-dev                         **failed**              1 error
 ~   ├─ aws-native:wafv2:WebACL  web-acl-no-description          **updating failed**     [diff: ~visibilityConfig]; 1 error
 ~   ├─ aws-native:wafv2:WebACL  web-acl-no-tags                 **updating failed**     [diff: ~visibilityConfig]; 1 error
 ~   ├─ aws-native:wafv2:WebACL  web-acl-no-tags-no-description  **updating failed**     [diff: ~visibilityConfig]; 1 error
 ~   └─ aws-native:wafv2:WebACL  web-acl                         updated                 [diff: ~visibilityConfig]

Diagnostics:
  pulumi:pulumi:Stack (waf-dev):
    error: update failed

  aws-native:wafv2:WebACL (web-acl-no-description):
    error: operation error CloudControl: UpdateResource, https response error StatusCode: 400, RequestID: 1c6219b8-25c6-4cfd-a1eb-888b03114aa9, api error ValidationException: Model validation failed (#/Description: failed validation constraint for keyword [pattern])

  aws-native:wafv2:WebACL (web-acl-no-tags):
    error: operation error CloudControl: UpdateResource, https response error StatusCode: 400, RequestID: d96d8ef1-9055-4562-808d-689a49ff32de, api error ValidationException: Model validation failed (#/Tags: expected minimum item count: 1, found: 0)

  aws-native:wafv2:WebACL (web-acl-no-tags-no-description):
    error: operation error CloudControl: UpdateResource, https response error StatusCode: 400, RequestID: c8e4dd3e-de87-452d-9584-9e00184fac5c, api error ValidationException: Model validation failed (#/Description: failed validation constraint for keyword [pattern]
    #/Tags: expected minimum item count: 1, found: 0)

Resources:
    ~ 1 updated
    1 unchanged

Duration: 12s

@viveklak I then added ignoreChanges: ["tags", "description"] and it did not make a difference.

I have updated the code in the example repo with the changes.

igor-eyrich commented 2 years ago

Exactly same here. ignore_changes doesn't do any difference. Is it planned to be fixed anytime soon?

viveklak commented 1 year ago

Apologies. IgnoreChanges is not the right suggestion. As demonstrated by @henriiik in https://github.com/pulumi/pulumi-aws-native/issues/415#issuecomment-1091063070 setting the description and some tag value at creation time avoids the overeager validation.

FWIW this is still a bug with cloudcontrol. The updates are converted to patch operations in cloudcontrol by the aws-native provider which are identical for resources with both tags and description specified and for those missing these fields, e.g.:

# Resource contains description and tags
 {"ClientToken":"XXXX","Identifier":"web-acl-9ddba5e|17ba60ad-23c4-49a8-8203-b0010b86a15d|REGIONAL","PatchDocument":"[{\"op\":\"replace\",\"path\":\"/VisibilityConfig\",\"value\":{\"CloudWatchMetricsEnabled\":false,\"MetricName\":\"1665549475007\",\"SampledRequestsEnabled\":false}}]","TypeName":"AWS::WAFv2::WebACL"}

# Resource doesn't contain description or tags
{"ClientToken":"XXXX","Identifier":"web-acl-no-tags-no-description-b4af193|a9091717-d347-4ed6-9b03-d6fee20e45c6|REGIONAL","PatchDocument":"[{\"op\":\"replace\",\"path\":\"/VisibilityConfig\",\"value\":{\"CloudWatchMetricsEnabled\":false,\"MetricName\":\"1665549474961\",\"SampledRequestsEnabled\":false}}]","TypeName":"AWS::WAFv2::WebACL"}

However, cloudcontrol's translation to wafv2 service endpoints seems to tickle the validation checks in latter but not the former.

Thanks @henriiik for the excellent repro. I am raising this again with our AWS contacts.

danielrbradley commented 1 year ago

According to our contacts at AWS this has now been addressed and should be rolled out to regions within the next 7-10 days.

Internal ref: 10998445201

SuryaTeja18 commented 1 year ago

This issue is still not addressed, error can still be reproduced.

joewragg commented 1 year ago

Please fix

danielrbradley commented 1 year ago

We have re-raised this with AWS and are awaiting a fix.

alextbok commented 1 month ago

This is still an issue. Any update on timeline to fix?

corymhall commented 2 weeks ago

I just tested this out and was unable to reproduce. The only error I received was related the the description field which is resolved if description is provided.

pulumi / pulumi-aws-native