Cloud armor rule update fails

[mikhailshilkov]

We wanted to update the IP whitelisting range for the cloud armor. However, any update of rules fails.

For example, see the code:

const edgeSecurityPolicy = new SecurityPolicy(name('sec-policy-edge', getSuffix()), {
        project: environment.project.gcp.projectId,
        description: 'Edge Security Policy',
        type: SecurityPolicyType.CloudArmorEdge,
        rules: [
            {
                action: 'allow',
                priority: 1000,
                match: {
                    versionedExpr: 'SRC_IPS_V1',
                    config: {
                        srcIpRanges: ipRanges,
                    },
                },
            },
            {
                action: 'deny(403)',
                priority: 2147483647,
                match: {
                    versionedExpr: 'SRC_IPS_V1',
                    config: {
                        srcIpRanges: ['*'],
                    },
                },
            },
        ],
    });

If you add more IP ranges in the array ipRanges, an update fails.

Steps to reproduce

1. Create a Google Cloud Armor security policy with source IP range
2. Update the source IP range to add more IPs

Expected: Update succeeds.

Actual:

error sending request: googleapi: Error 400: Invalid value for field 'resource': 
'{  "name": "bla-4a6a8c1",  "description": "Edge Security Policy",  "rule": [{    
"priority": 1000,  ...'. Rules cannot be updated with patch, please use addRule, 
removeRule, or patchRule instead.
...

[sghosh-discovery]

I used a workaround by adding a random string at the end of the name before creating the security policy. So every deployment creates a new policy and deletes the old.

[mikhailshilkov]

@sghosh-discovery thank you for confirming the workaround. Our most likely short-term fix will do the same automatically.

[hjorth]

Any status on this issues? Having the same issue.

Using random names for the policy - is not an option - When you combine it with GKE + Ingress resource. (Or it makes it really hard - at least).

[leezen]

An alternative to adding a random string is to use replaceOnChanges(see https://www.pulumi.com/blog/pulumi-release-notes-m58/#new-replaceonchanges-resource-option)

[marvmariano]

replaceOnChanges will not work though if the Cloud Armor policy is already in used. It will throw below error. Removing the resource (e.g. GKE Ingress Load Balancers) from Cloud Armor policy might not be an ideal solution especially if you have this already in production environment. Does anyone have other approach in updating Cloud Armor policy if it's already in used?

operation errored with map[errors:[map[code:RESOURCE_IN_USE_BY_ANOTHER_RESOURCE message:The security_policy resource 'projects/demo-app-qa/global/securityPolicies/demo-qa-cloudarmor-allow-policy-classic-de44ca2' is already being used by 'projects/demo-app-qa/global/backendServices/k8s-be-32448--b027567ac73d16b6']]]

[britwilb245]

@marvmariano you need to manually remove the previous policy on your cloud armor instance and run the policy again. Or fall back to the Classic provider (which does not have this issue occurring).

I also had the the same error occur in my instance which appears to be due to this same issue.

[marvmariano]

@marvmariano you need to manually remove the previous policy on your cloud armor instance and run the policy again. Or fall back to the Classic provider (which does not have this issue occurring).

I also had the the same error occur in my instance which appears to be due to this same issue.

Yeah, I've tested in Classic as well and no issue with that. That will be the solution for now. With native, removing the previous policy or the target (if in used) is not ideal solution if you already running in production especially in highly-regulated environment.