google_compute_subnetwork enable_flow_logs deprecated in google terraform 3.0.0 and GCPNetworkEnableFlowLogsConstraintV1 broken

[jsmilani]

I am applying the the https://github.com/terraform-google-modules/terraform-example-foundation for my company and used this policy library but encountered a problem when validating the 3-networks stage. We are using the Cloud Build instructions so we never customized terraform or google plugin versions at all. We are using most, if not all the policies from this repo because we need the added security for compliance reasons. The error we are getting is:

Constraint GCPNetworkEnableFlowLogsConstraintV1.enable_network_flow_logs on resource //compute.googleapis.com/projects/prj-c-base-net-hub-32b0/regions/us-central1/subnetworks/sb-c-shared-base-hub-us-central1: Flow logs are disabled in subnetwork //compute.googleapis.com/projects/prj-c-base-net-hub-32b0/regions/us-central1/subnetworks/sb-c-shared-base-hub-us-central1.

The issue is that subnet has logging enabled so it shouldn't be failing validation:

# module.base_shared_vpc[0].module.main.module.subnets.google_compute_subnetwork.subnetwork["us-central1/sb-c-shared-base-hub-us-central1"] will be created
+ resource "google_compute_subnetwork" "subnetwork" {
    + creation_timestamp         = (known after apply)
    + description                = "Base network hub subnet for us-central1"
    + external_ipv6_prefix       = (known after apply)
    + fingerprint                = (known after apply)
    + gateway_address            = (known after apply)
    + id                         = (known after apply)
    + ip_cidr_range              = "10.0.0.0/24"
    + ipv6_cidr_range            = (known after apply)
    + name                       = "sb-c-shared-base-hub-us-central1"
    + network                    = "vpc-c-shared-base-hub"
    + private_ip_google_access   = true
    + private_ipv6_google_access = (known after apply)
    + project                    = "prj-c-base-net-hub-32b0"
    + purpose                    = (known after apply)
    + region                     = "us-central1"
    + secondary_ip_range         = []
    + self_link                  = (known after apply)
    + stack_type                 = (known after apply)

    + log_config {
        + aggregation_interval = "INTERVAL_5_SEC"
        + filter_expr          = "true"
        + flow_sampling        = 0.5
        + metadata             = "INCLUDE_ALL_METADATA"
      }
  }

My best guess is the line https://github.com/GoogleCloudPlatform/policy-library/blob/master/policies/templates/gcp_network_enable_flow_logs_v1.yaml#L62 is reading enableFlowLogs from the resource data but the enable_flow_logs field is deprecated according to https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/version_3_upgrade#enable_flow_logs-is-now-removed

enable_flow_logs has been removed and should be replaced by the log_config block with configurations for flow logging. Enablement of flow logs is now controlled by whether log_config is defined or not instead of by the enable_flow_logs variable. Users with enable_flow_logs = false only need to remove the field.

I assume there is a way to create my own GCPNetworkEnableFlowLogsConstraintV2 to fix this or more logic to check plugin versions, but I am new to terraform-validator and policies and these google golang libraries so it isn't obvious how to update the rego to fix it.

[jsmilani]

I found this source code https://github.com/GoogleCloudPlatform/terraform-google-conversion/blob/master/google/compute_subnetwork.go#L261 that helped me locate an enabled field nested under logConfig that should be equivalent to the deprecated enable_flow_logs field:

diff --git a/policies/templates/gcp_network_enable_flow_logs_v1.yaml b/policies/templates/gcp_network_enable_flow_logs_v1.yaml
index b331f83..fee5894 100644
--- a/policies/templates/gcp_network_enable_flow_logs_v1.yaml
+++ b/policies/templates/gcp_network_enable_flow_logs_v1.yaml
@@ -59,7 +59,8 @@ spec:
                asset.asset_type == "compute.googleapis.com/Subnetwork"

                network := asset.resource.data
-               enable_flow_logs := lib.get_default(network, "enableFlowLogs", false)
+               log_config := lib.get_default(network, "logConfig", {})
+               enable_flow_logs := lib.get_default(log_config, "enable", false)
                enable_flow_logs == false

                message := sprintf("Flow logs are disabled in subnetwork %v.", [asset.name])

I tested with log_config set and unset. The violation is raised when unset so this looks like a solution for the newer google terraform libs. I assume this belongs in a new V2 version of the old template policies/templates/gcp_network_enable_flow_logs_v2.yaml ?

[morgante]

@jsmilani Thanks for investigating. I think we can actually do this without forcing a new policy version by:

1. Looking for the old field (enableFlowLogs)
2. Looking for the new field (logConfig)

Then, raise a violation if neither is found.

If you're able to work on a PR for this, it would be very appreciated!

[DevOpsCoder15]

I see this bug is still open although a solution has been identified. When will the fix be applied?