Error updating BackendService: googleapi: Error 400: Invalid value for field 'resource.securitySettings': ''. Security settings is not supported., invalid

[rumblersoppa]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.6.6 on darwin_amd64

• provider registry.terraform.io/hashicorp/google v5.10.0
• provider registry.terraform.io/hashicorp/null v3.2.2
• provider registry.terraform.io/hashicorp/random v3.6.0
• provider registry.terraform.io/hashicorp/time v0.10.0

Affected Resource(s)

• google_compute_backend_service

Terraform Configuration Files

#### Backend Service
resource "google_compute_backend_service" "global_backend_service_assinatura_website" {
  name                  = google_storage_bucket.storage_assinatura_website.name
  project               = var.project_id
  description           = "Global Backend Service for Assinatura Website"
  load_balancing_scheme = "EXTERNAL_MANAGED"
  protocol              = "HTTPS"
  session_affinity      = "NONE"
  timeout_sec           = 60
  locality_lb_policy    = "ROUND_ROBIN"
  enable_cdn            = false 

  backend {
    group           = google_compute_global_network_endpoint_group.global_neg_assinatura.self_link
    balancing_mode  = "UTILIZATION"
    capacity_scaler = 1.0
  }

  custom_request_headers = [
    "host:${google_storage_bucket.storage_assinatura_website.name}",
    "cookie:",
  ]

  custom_response_headers = var.custom_response_headers

  edge_security_policy = google_compute_security_policy.edge_secure_policy_assinatura.name

  # security_settings {
  #   client_tls_policy = ""
  #   subject_alt_names = []
  # }

  lifecycle {
    ignore_changes = [
      security_settings,
    ]
  }
}

#### Update Backend Service
resource "null_resource" "update_backend_service_assinatura_website" {
  triggers = {
    backend_service = google_compute_backend_service.global_backend_service_assinatura_website.id
  }
  depends_on = [google_compute_backend_service.global_backend_service_assinatura_website]
  provisioner "local-exec" {
    command = <<-EOT
      curl -X PATCH \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $(gcloud auth print-access-token)" \
      -d '{
        "securitySettings": {
          "awsV4Authentication": {
            "accessKeyId": "${google_storage_hmac_key.private_access_gcs_hmac_key.access_id}",
            "accessKey": "${google_storage_hmac_key.private_access_gcs_hmac_key.secret}",
            "originRegion": "${var.project_region}"
          }
        }
      }' \
      "https://compute.googleapis.com/compute/v1/projects/${var.project_id}/global/backendServices/${google_compute_backend_service.global_backend_service_assinatura_website.name}"
    EOT
  }
}

Debug Output

https://gist.github.com/rumblersoppa/8fa6304680c31b2f49912538c6ebf05c

Expected Behavior

I hope that when I make changes to the google_compute_backend_service resource and apply those changes using Terraform, the resource will update accordingly. Specifically, removing the security_settings block from the configuration should cause Terraform to ignore changes related to this block during the apply process.

Actual Behavior

I followed the recommended steps, including using the lifecycle block with ignore_changes for the security_settings block. Despite these efforts, Terraform is still trying to apply changes to the security_settings block, leading to an error during the application process.

Important Factoids

• The security_settings block is dynamically modified by a null_resource that executes a API call to Google Cloud. This call adds the security_settings block with the awsV4Authentication argument and its associated values.

• I am aware that this is an advanced configuration and not be officially supported by the current version of the Google Cloud provider for Terraform.

References

https://cloud.google.com/cdn/docs/configure-private-origin-authentication?hl=pt-br#gcloud

https://developer.hashicorp.com/terraform/language/meta-arguments/lifecycle

[edwardmedia]

@rumblersoppa help me to understand, the method used in the config is PATCH while it is PUT in the debug. Was the log from that execution? Did I miss something? In the local-exec command, I do see you provided a map data to securitySettings, but not sure why it is empty showing in the log. Instead of local-exec, are you able to make a update via gcloud? The issue seems to me that it is about how to call the api. There is not much about the Terraform provider. Do you agree?

[rumblersoppa]

@rumblersoppame ajude a entender, o método usado na configuração é PATCH enquanto é PUT no debug. O log dessa execução foi? Perdi algo? No comando local-exec , vejo que você forneceu dados de mapa para securitySettings , mas não tenho certeza por que eles estão vazios aparecendo no log. Em vez de local-exec , você consegue fazer uma atualização via gcloud? A questão me parece ser como chamar a API. Não há muito sobre o provedor Terraform. Você concorda?

Hello @edwardmedia , This is exactly the point that made me open this issue, I created the resource backend service and then null_resource makes a call to the Google api with the PATCH method to add the securitySettings configuration, in the terraform GCP module documentation the compute_backend_service resource has the security_settings block which has only two arguments: client_tls_policy and subject_alt_names, but There is already a configuration that is still only done via command gcloud or call api to add the content:

securitySettings:
   awsV4Authentication:
     accessKeyId: ACCESS_KEY_ID
     accessKey: ACCESS_KEY
     [accessKeyVersion: ACCESS_KEY_VERSION]
     originRegion: REGION

I may have misunderstood the lifecycle use case with ignore_changes, the intention was to create the backend service resource with terraform, as it does not have the awsV4Authentication parameter in the module's security_settings block, add the parameters missing using an api call, and using the lifecycle ingore_changes to not touch the resource's security_settings block. But terraform seems to ignore this premise and when running a plan again to add another resource, it touches secure_settings, and when running an apply it generates exactly the error reported in the gist attached to the issue.

[edwardmedia]

@rumblersoppa If I understand correctly, you are talking about null_resource. Either gcloud or call api, you bypass the terraform resource. Still not see how we can help from the provider perspective. Does it make sense?

gcloud or call api to add the content:

[rumblersoppa]

@rumblersoppaSe bem entendi, você está falando de null_resource. Seja gcloud ou call api, você ignora o recurso terraform. Ainda não vejo como podemos ajudar do ponto de vista do fornecedor. Isso faz sentido?

gcloud or call api to add the content:

Hello @edwardmedia,

I created a repository with a complete example so that you can test the behavior directly.

GitHub Repository - terraform_live_cycle_debug

[edwardmedia]

@hao-nan-li what do you see about this issue?

[hao-nan-li]

Were you able to get the same error if you use a HTTP request to make the update instead of using Terraform?

[rumblersoppa]

Você conseguiu o mesmo erro se usar uma solicitação HTTP para fazer a atualização em vez de usar o Terraform?

No, when using an HTTP request, I do not receive the error.

[hao-nan-li]

If that's the case to make the update, is it still necessary to use TF to make the update?

[markmssd]

I am also having the same issue (on TF v1.8.0). It's worth noting that the issue does not lay into the usage of null_resource. It can also be reproduced by:

1. Set up a valid google_compute_backend_service via Terraform
2. Update the backend service securitySettings. awsV4Authentication, without using null_resource, but simply gcloud, following Google CDN official docs
3. Try to update the google_compute_backend_service again (change any field like description for example

It will throw the same error as OP. This happens regardless of the usage of ignore_changes. I've tried it with

ignore_changes = [
  security_settings,
  # or
  security_settings["awsV4Authentication"],
  # or
  security_settings["aws_v4_authentication"],
]

Ideally, the google_compute_backend_service should have support for the aws_v4_authentication attribute, just like google_network_services_edge_cache_origin does (AFAIK the latter is used for Media CDN, not for a Cloud CDN, thus we can't use it).

A relevant issue is also https://github.com/hashicorp/terraform-provider-google/issues/12862, where aws_v4_authentication support was added to Media CDN.

Does that make sense? I can attempt a PR if it does!

[gtzo-anchorage]

Also facing this same issue.

[RafaelHGBotelho]

Same issue here, I had to export the backend service and import it again to provision via gcloud.

[ankitjethi]

Facing the same issue.

[Aiden0]

Same issue here. I do have a kinda work around which is to just trigger the the null resource everytime i.e

resource "null_resource" "update_backend_service" {
  triggers = {
    always_recreate = uuid()
  }
  ...

This still might be a race condtion, might be able to trigger on whenever backend service security_settings gets changed, might be more robust.