Cannot add server TLS policy to existing google_compute_region_target_https_proxy

gservat commented 1 month ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to a user, that user is claiming responsibility for the issue.
Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.9.2 on arm64

provider registry.terraform.io/hashicorp/google v5.38.0
provider registry.terraform.io/hashicorp/google-beta v5.38.0

Affected Resource(s)

google_compute_region_target_https_proxy

Terraform Configuration

Initial Terraform configuration:

resource "google_compute_forwarding_rule" "default-https" {
  region                = "us-central1"
  name                  = "https-frwd-rule"
  load_balancing_scheme = "INTERNAL_MANAGED"
  target                = google_compute_region_target_https_proxy.default-https.self_link
  network               = google_compute_network.ilb_network.name
  subnetwork            = google_compute_subnetwork.ilb_subnet.name
  ip_address            = google_compute_address.consumer_address.id
  ip_protocol           = "TCP"
  port_range            = "443"
  allow_global_access   = "true"
  depends_on            = [google_compute_subnetwork.ilb_subnet2]
}

resource "google_compute_region_backend_service" "default" {
  region                = "us-central1"
  name                  = "backend-service"
  protocol              = "HTTPS"
  port_name             = "https-server"
  load_balancing_scheme = "INTERNAL_MANAGED"
  session_affinity      = "HTTP_COOKIE"
  health_checks         = [google_compute_region_health_check.default.self_link]
  locality_lb_policy    = "RING_HASH"

  consistent_hash {
    http_cookie {
      ttl {
        # 24hr cookie ttl
        seconds = 86400
        nanos   = null
      }
      name = "X-CLIENT-SESSION"
      path = null
    }
    http_header_name  = null
    minimum_ring_size = 1024
  }

  log_config {
    enable      = true
    sample_rate = 1.0
  }
}

resource "google_compute_region_health_check" "default" {
  region              = "us-central1"
  name                = "hc"
  timeout_sec         = 5
  check_interval_sec  = 30
  healthy_threshold   = 3
  unhealthy_threshold = 3

  https_health_check {
    port         = 443
    request_path = "/health"
  }
}

resource "google_compute_region_target_https_proxy" "default-https" {
  region           = "us-central1"
  name             = "https-proxy"
  url_map          = google_compute_region_url_map.default-https.self_link
  ssl_certificates = [google_compute_region_ssl_certificate.foobar0.self_link]
}

resource "google_compute_region_url_map" "default-https" {
  region          = "us-central1"
  name            = "lb"
  default_service = google_compute_region_backend_service.default.id
}

resource "google_compute_region_ssl_certificate" "foobar0" {
  name        = "httpsproxy-test-cert0"
  description = "very descriptive"
  private_key = file("test.key")
  certificate = file("test.crt")
}

resource "google_compute_network" "ilb_network" {
  name                    = "tf-test-l4-ilb-network"
  auto_create_subnetworks = false
}

resource "google_compute_subnetwork" "ilb_subnet" {
  name          = "tf-test-l4-ilb-subnet"
  ip_cidr_range = "10.0.1.0/24"
  region        = "us-central1"
  network       = google_compute_network.ilb_network.id
}

resource "google_compute_subnetwork" "ilb_subnet2" {
  name          = "tf-test-l4-ilb-subnet2"
  ip_cidr_range = "10.142.0.0/20"
  region        = "us-central1"
  purpose       = "REGIONAL_MANAGED_PROXY"
  role          = "ACTIVE"
  network       = google_compute_network.ilb_network.id
}

resource "google_compute_address" "consumer_address" {
  name         = "tf-test-website-ip"
  region       = "us-central1"
  subnetwork   = google_compute_subnetwork.ilb_subnet.id
  address_type = "INTERNAL"
}

Subsequent update to enable server TLS policy for mTLS:

data "google_project" "project" {}

resource "google_compute_forwarding_rule" "default-https" {
  region                = "us-central1"
  name                  = "https-frwd-rule"
  load_balancing_scheme = "INTERNAL_MANAGED"
  target                = google_compute_region_target_https_proxy.default-https.self_link
  network               = google_compute_network.ilb_network.name
  subnetwork            = google_compute_subnetwork.ilb_subnet.name
  ip_address            = google_compute_address.consumer_address.id
  ip_protocol           = "TCP"
  port_range            = "443"
  allow_global_access   = "true"
  depends_on            = [google_compute_subnetwork.ilb_subnet2]
}

resource "google_compute_region_backend_service" "default" {
  region                = "us-central1"
  name                  = "backend-service"
  protocol              = "HTTPS"
  port_name             = "https-server"
  load_balancing_scheme = "INTERNAL_MANAGED"
  session_affinity      = "HTTP_COOKIE"
  health_checks         = [google_compute_region_health_check.default.self_link]
  locality_lb_policy    = "RING_HASH"

  consistent_hash {
    http_cookie {
      ttl {
        # 24hr cookie ttl
        seconds = 86400
        nanos   = null
      }
      name = "X-CLIENT-SESSION"
      path = null
    }
    http_header_name  = null
    minimum_ring_size = 1024
  }

  log_config {
    enable      = true
    sample_rate = 1.0
  }
}

resource "google_compute_region_health_check" "default" {
  region              = "us-central1"
  name                = "hc"
  timeout_sec         = 5
  check_interval_sec  = 30
  healthy_threshold   = 3
  unhealthy_threshold = 3

  https_health_check {
    port         = 443
    request_path = "/health"
  }
}

resource "google_compute_region_target_https_proxy" "default-https" {
  region           = "us-central1"
  name             = "https-proxy"
  url_map          = google_compute_region_url_map.default-https.self_link
  ssl_certificates = [google_compute_region_ssl_certificate.foobar0.self_link]
  server_tls_policy = google_network_security_server_tls_policy.default.id
}

resource "google_certificate_manager_trust_config" "default" {
  provider = google-beta

  location    = "us-central1"
  name        = "trust-config"

  trust_stores {
    trust_anchors {
      pem_certificate = file("ca_cert.pem")
    }
    intermediate_cas {
      pem_certificate = file("ca_cert.pem")
    }
  }
}

resource "google_network_security_server_tls_policy" "default" {
  provider = google-beta

  location   = "us-central1"
  name       = "tls-policy"
  allow_open = "false"
  mtls_policy {
    client_validation_mode = "REJECT_INVALID"
    client_validation_trust_config = "projects/${data.google_project.project.number}/locations/us-central1/trustConfigs/${google_certificate_manager_trust_config.default.name}"
  }
}

resource "google_compute_region_url_map" "default-https" {
  region          = "us-central1"
  name            = "lb"
  default_service = google_compute_region_backend_service.default.id
}

resource "google_compute_region_ssl_certificate" "foobar0" {
  name        = "httpsproxy-test-cert0"
  description = "very descriptive"
  private_key = file("test.key")
  certificate = file("test.crt")
}

resource "google_compute_network" "ilb_network" {
  name                    = "tf-test-l4-ilb-network"
  auto_create_subnetworks = false
}

resource "google_compute_subnetwork" "ilb_subnet" {
  name          = "tf-test-l4-ilb-subnet"
  ip_cidr_range = "10.0.1.0/24"
  region        = "us-central1"
  network       = google_compute_network.ilb_network.id
}

resource "google_compute_subnetwork" "ilb_subnet2" {
  name          = "tf-test-l4-ilb-subnet2"
  ip_cidr_range = "10.142.0.0/20"
  region        = "us-central1"
  purpose       = "REGIONAL_MANAGED_PROXY"
  role          = "ACTIVE"
  network       = google_compute_network.ilb_network.id
}

resource "google_compute_address" "consumer_address" {
  name         = "tf-test-website-ip"
  region       = "us-central1"
  subnetwork   = google_compute_subnetwork.ilb_subnet.id
  address_type = "INTERNAL"
}

Debug Output

No response

Expected Behavior

The update to enable server_tls_policy on the target HTTPS proxy applies cleanly and does not re-create the target HTTPS proxy as this is supported by Google's API to PATCH the resource.

Actual Behavior

It attempts to recreate the HTTPS proxy but fails because there is a forwarding rule attached to the proxy:

Error: Error when reading or editing RegionTargetHttpsProxy: googleapi: Error 400: The target_https_proxy resource 'projects/<project>/regions/us-central1/targetHttpsProxies/https-proxy' is already being used by 'projects/<project>/regions/us-central1/forwardingRules/https-frwd-rule', resourceInUseByAnotherResource

Steps to reproduce

terraform apply original TF config
terraform apply updated TF config with the server TLS policy

Important Factoids

No response

References

Similar issue to https://github.com/hashicorp/terraform-provider-google/issues/16963 except this concerns the server_tls_policy vs ssl_policy.

b/355116824

b/355124984

ggtisc commented 1 month ago

Confirmed issue!

If the google_compute_region_target_https_proxy resource already exists after create or use the google_network_security_server_tls_policy resource in the current terraform configuration then returns the next error message:

Error: Error when reading or editing RegionTargetHttpsProxy: googleapi: Error 400: The target_https_proxy resource 'projects/<project>/regions/us-central1/targetHttpsProxies/https-proxy' is already being used by 'projects/<project>/regions/us-central1/forwardingRules/https-frwd-rule', resourceInUseByAnotherResource

github-actions[bot] commented 2 days ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-google