google cloud functions cannot get updates on the build_environment_variables

leandrogoe commented 1 month ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to a user, that user is claiming responsibility for the issue.
Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

Terraform v1.4.0 on darwin_arm64

provider registry.terraform.io/hashicorp/archive v2.2.0
provider registry.terraform.io/hashicorp/google v4.30.0
provider registry.terraform.io/hashicorp/google-beta v4.30.0
provider registry.terraform.io/hashicorp/random v3.6.2

Affected Resource(s)

google_cloudfunctions_function

Terraform Configuration

variable "some_variable" {
  description = "variable required at build time"
}

resource "google_storage_bucket" "google_functions_bucket" {
  name     = "${terraform.workspace}-functions"
  location = "US"
}

data "archive_file" "my_function" {
  type        = "zip"
  source_dir  = "${path.root}/cloud_functions/my_function"
  output_path = "/tmp/my_function.zip"
}

resource "google_storage_bucket_object" "cache_my_function" {
  name   = "${data.archive_file.my_function.output_md5}.zip"
  bucket = google_storage_bucket.google_functions_bucket.name
  source = "/tmp/my_function.zip"
}

resource "google_cloudfunctions_function" "my_simple_function" {
  name        = "my-simple-function"
  runtime     = "nodejs18"

  available_memory_mb          = 128
  source_archive_bucket        = google_storage_bucket.google_functions_bucket.name
  source_archive_object        = google_storage_bucket_object.cache_my_function.name
  trigger_http                 = true
  https_trigger_security_level = "SECURE_ALWAYS"
  timeout                      = 300
  entry_point                  = "someMethod"

  build_environment_variables = {
    SOME_ENV_VAR = var.some_variable
  }
}

Debug Output

No response

Expected Behavior

When changing the value of any of build_environment_variables terraform should be able to handle it properly

Actual Behavior

When changing the value of any of build_environment_variables for an EXISTING cloud function terraform fails:

Error: Error while updating cloudfunction configuration: googleapi: Error 400: The request has errors
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.BadRequest",
│     "fieldViolations": [
│       {
│         "description": "build environment variables are not supported by this runtime",
│         "field": "build_environment_variables"
│       }
│     ]
│   }
│ ]
│ , badRequest
│ 
│   with module.pgp_decryption.google_cloudfunctions_function.my_simple_function,
│   on main.tf line 17, in resource "google_cloudfunctions_function" "my_simple_function":
│   17: resource "google_cloudfunctions_function" "my_simple_function" {
│

Looking at the logs, it appears that there is a problem with the PATCH method of the Google API. Whenever you try to update the build_environment_variables you also need to include the runtime in the update mask for it to succeed.

Steps to reproduce

terraform apply -var="some_variable=1"
terraform apply -var="some_variable=2"

The first apply succeeds, but the second one fails with the error above.

Important Factoids

No response

References

No response

b/343970055

ggtisc commented 1 month ago

Confirmed issue!

After creating the resources each attempt to change the variable value fails with the next error:

Error: Error while updating cloudfunction configuration: googleapi: Error 400: The request has errors
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.BadRequest",
│     "fieldViolations": [
│       {
│         "description": "build environment variables are not supported by this runtime",
│         "field": "build_environment_variables"
│       }
│     ]
│   }
│ ]
│ , badRequest
│ 
│   with module.pgp_decryption.google_cloudfunctions_function.my_simple_function,
│   on main.tf line 17, in resource "google_cloudfunctions_function" "my_simple_function":
│   17: resource "google_cloudfunctions_function" "my_simple_function" {

szwalkowski commented 1 month ago

This is a critical issue.

The google_cloudfunctions_function resource has become unreliable in certain scenarios. It appears that GCP is doing a partial rollout, as this problem does not affect all environments yet. Many pipelines are breaking, but inconsistently across environments. This is particularly important for deploying specific versions of functions in these environments.

Does anyone have any official information from GCP about this change?

greenozon commented 3 weeks ago

Absolutely critical issue! hit this today in prod env, that was crazy insane... error and error and error... what is going on, Google???

greenozon commented 3 weeks ago

after multi-hours of stupid debugging the workaround was to degrade runtime and then put it back but why???

mcompen commented 2 weeks ago

Same error message received with python39/310 runtime. Note that this is probably an issue unrelated to this provider, since the same error message pops up in Console.

greenozon commented 2 weeks ago

This is very tricky and intermittent (floating) issue.. I hit it once (see above) and then it's gone... running same tf scripts/env/vars not able to reproduce anymore it means this is smth Google-related and not a permanent issue, unfortunately

pdobrzanski-appfire commented 2 weeks ago

I got the information from GCP support that bugfix will roll out next week, so fingers crossed. This is indeed not a provider bug.

hashicorp / terraform-provider-google