Support serverless neg in google_compute_region_backend_service

bahag-klickst commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

We are using the public preview feature of Internal Load Balancer supporting Cloud Run backends (https://cloud.google.com/load-balancing/docs/l7-internal/setting-up-l7-internal-serverless) and created a bash script for invoking the gcloud commands. We now wanted to build the setup via terraform, but unfortunately it fails due to google_compute_region_backend_service not supporting serverless negs. We are receiving the following error:

╷
│ Error: Error creating RegionBackendService: googleapi: Error 400: Invalid value for field 'resource.backends[0].balancingMode': 'CONNECTION'. Balancing mode is not supported for Serverless network endpoint groups., invalid
│ 
│   with google_compute_region_backend_service.ilb_cr_backend_service,
│   on ilb.tf line 54, in resource "google_compute_region_backend_service" "ilb_cr_backend_service":
│   54: resource "google_compute_region_backend_service" "ilb_cr_backend_service" {
│ 
╵

New or Affected Resource(s)

google_compute_region_backend_service

Potential Terraform Configuration

resource "google_cloud_run_service" "ilb_cr_service" {
  provider = google-beta
  project  = var.project
  location = var.region
  name     = "ilb-cr"
  template {
    spec {
      containers {
        image = "gcr.io/cloudrun/hello"
      }
    }
    metadata {
      annotations = {
        # Limit scale up to prevent any cost blow outs!
        "autoscaling.knative.dev/maxScale" = "5"
        # Use the VPC Connector
        "run.googleapis.com/vpc-access-connector" = "serverless-vpc-connector"
        # all egress from the service should go through the VPC Connector
        "run.googleapis.com/vpc-access-egress" = "all-traffic"
      }
    }
  }
  traffic {
    percent         = 100
    latest_revision = true
  }
  metadata {
    annotations = {
      # Internal traffic only
      "run.googleapis.com/ingress" = "internal"
    }
  }
}

resource "google_compute_region_network_endpoint_group" "ilb_cr_neg" {
  project               = var.project
  region                = var.region
  name                  = "ilb-cr-neg"
  network_endpoint_type = "SERVERLESS"
  cloud_run {
    service = google_cloud_run_service.ilb_cr_service.name
  }
}

resource "google_compute_region_backend_service" "ilb_cr_backend_service" {
  project               = var.project
  region                = var.region
  name                  = "ilb-cr-backend-service"
  protocol              = "HTTPS"
  load_balancing_scheme = "INTERNAL_MANAGED"
  backend {
    group = google_compute_region_network_endpoint_group.ilb_cr_neg.id
  }
}

References

In the docs it states, that when using the gcloud commands, one should use the following commands to setup the backend service and attach the neg:

gcloud compute backend-services create BACKEND_SERVICE_NAME \
        --load-balancing-scheme=INTERNAL_MANAGED \
        --protocol=HTTP \
        --region=REGION

gcloud compute backend-services add-backend BACKEND_SERVICE_NAME \
        --region=REGION \
        --network-endpoint-group=SERVERLESS_NEG_NAME \
        --network-endpoint-group-region=REGION

So I guess backend block has to be expanded to support neg.

b/314616920

bahag-klickst commented 2 years ago

Okay, just found out, that I missed to check setting balancing_mode to "UTILIZATION". Using "UTILIZATION" works and setup of ILB with CR based backend is possible.

bahag-klickst commented 2 years ago

However, not sure if this is the right setting.

googlerjk commented 2 years ago

I reviewed the latest Magic Modules resource "google_cloud_run_service" in the regional neg and see not all required fields are supported and/or tested.

The cloud_run_service.traffic property is not supported in the terraform.yaml. Also, the full configuration expressed here is not represented or tested in its entirety. Once the missing fields and handwritten tests are added, the support for serverless neg should done.

bahag-klickst commented 1 year ago

Any updates on this issue?

sudermanjr commented 1 year ago

Also would love to see this feature implemented

pawelJas commented 4 months ago

Any updates on this issue? @bahag-klickst you have found the correct workaround. Default Terraform value is different than deafult GCP value, setting it explicitly solves the problem.

Changing the default value is a breaking change, so it will need to wait for the next.

@melinath Could you mark this bug as breaking change, please? Could you also close 17257 as duplicate?

melinath commented 3 months ago

@hao-nan-li could you look into making the changes requested in https://github.com/hashicorp/terraform-provider-google/issues/12299#issuecomment-2132987091 if appropriate?

pawelJas commented 3 months ago

@melinath @hao-nan-li I am planning to preapre a PR which clears the deafult values next week. Making 17257 duplicate could help, but we can just mark both as resolved after the PR.

hao-nan-li commented 3 months ago

Thanks @pawelJas, just marked duplicate on #17257

pawelJas commented 2 months ago

The default value for balancing_mode in google_compute_region_backend_service has been changed to UTILIZATION (default GCP value) in version 6.0.0. That should resolve the serverless NEG support.

github-actions[bot] commented 1 month ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-google