Error assigning secrets to cloud functions2

assertnotnull commented 11 months ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

1.6.1

Affected Resource(s)

Google provider version 5.3.0

google_cloudfunctions2_function

Terraform Configuration Files

locals {
  function_file_path = "folder/functions.zip"
  environment_variables = {
    "NODE_ENV"                         = "production" 
  }
  secrets = {
    "APP_SECRET"        = "xxxxx
    "SENDGRID_API_KEY"  = "SG.1234"
    "TWILIO_AUTH_TOKEN" = "xxxxxxxxxxxxx"
    "TWILIO_SID"        = "xxxxxxxxxxxx"
  }
  secret_ids = {
    for name, secret in google_secret_manager_secret.secrets : name => secret.secret_id
  }
}

resource "google_secret_manager_secret" "secrets" {
  for_each  = local.secrets
  secret_id = each.key

  replication {
    user_managed {
      replicas {
        location = var.location
      }
    }
  }
}

resource "google_secret_manager_secret_version" "content" {
  for_each    = local.secrets
  secret      = google_secret_manager_secret.secrets[each.key].id
  secret_data = each.value
}

output "secrets" {
  value = local.secret_ids
}

module "fn_app_notification" {
  source                 = "../../module/functions-pubsub"
  location               = var.location
  function_name          = "${var.environment}-appNotification"
  function_entry_point   = "appNotification"
  topic_name             = "${var.environment}.app-notification"
  function_bucket        = var.serverless_bucket_name
  function_file_path     = local.function_file_path
  function_env_vars      = local.environment_variables
  runtime                = "nodejs18"
  compiled_zip_file_path = local.compiled_zip_file_path
  secrets                = local.secret_ids
}

the module

variable "topic_name" {
  type = string
}

variable "location" {
  type = string
}

variable "function_name" {
  type = string
}

variable "runtime" {
  type    = string
  default = "nodejs16"
}

variable "function_entry_point" {
  type        = string
  description = "The lambda function entry point"
}

variable "function_bucket" {
  type        = string
  description = "Which Google cloud storage bucket contains the function code"
}

variable "function_file_path" {
  type        = string
  description = "The path to the function code inside the bucket"
}

variable "compiled_zip_file_path" {
  type = string
}

variable "function_env_vars" {
  type = map(string)
}

variable "secrets" {
  type = map(string)
}

resource "google_storage_bucket_object" "source" {
  name   = var.function_file_path
  source = var.compiled_zip_file_path
  bucket = var.function_bucket
}

data "google_project" "project" {}
data "google_pubsub_topic" "topic" {
  name = var.topic_name
}

resource "google_cloudfunctions2_function" "function" {
  name     = var.function_name
  location = var.location

  build_config {
    runtime     = var.runtime
    entry_point = var.function_entry_point
    source {
      storage_source {
        bucket = var.function_bucket
        object = var.function_file_path
      }
    }
    environment_variables = {}
  }

  service_config {
    all_traffic_on_latest_revision   = true
    available_cpu                    = "0.1666"
    available_memory                 = "256M"
    environment_variables            = var.function_env_vars
    ingress_settings                 = "ALLOW_INTERNAL_ONLY"
    max_instance_count               = 100
    max_instance_request_concurrency = 1
    min_instance_count               = 0
    service                          = lower("${data.google_project.project.id}/locations/${var.location}/services/${var.function_name}")
    timeout_seconds                  = 60

    dynamic "secret_environment_variables" {
      for_each = var.secrets

      content {
        key        = secret_environment_variables.key
        project_id = data.google_project.project.id
        version    = "latest"
        secret     = secret_environment_variables.value
      }
    }
  }

  event_trigger {
    trigger_region = var.location
    event_type     = "google.cloud.pubsub.topic.v1.messagePublished"
    pubsub_topic   = data.google_pubsub_topic.topic.id
    retry_policy   = "RETRY_POLICY_DO_NOT_RETRY"
  }
}

Debug Output

Debug logs of interest: https://gist.github.com/assertnotnull/c7644085e2be407d0243835d69e12a0d

Error: Error updating function "projects/redacted/locations/northamerica-northeast1/functions/function-name": googleapi: Error 400: Could not update Cloud Run service projects/redacted/locations/northamerica-northeast1/services/function-name. service.spec.template.metadata.annotations[run.googleapis.com/secrets]: secrets annotation should have the format :projects/|/secrets/[,:projects/|/secrets/,...]

Expected Behavior

Update the secrets of the google function v2

Actual Behavior

Error with the above

Steps to Reproduce

terraform apply

b/307707144

josueetcom commented 10 months ago

Hi @assertnotnull , I'm looking at your debug log and it looks like in the projectId in the secret environment variable is prefixed with projects/. The documentation example however just has the project ID. Can you try removing that prefix?

assertnotnull commented 10 months ago

@josueetcom it gives the same error setting the project_id to data.google_project.project.project_id and then again when trying with the key set to "projects/${data.google_project.project.project_id}/secrets/${secret_environment_variables.key}" or the other formats in the error message

josueetcom commented 10 months ago

Sorry, I should've been clearer. What I mean is that you have these lines:

    dynamic "secret_environment_variables" {
        ...
        project_id = data.google_project.project.id
        ...
    }

Which is showing up in the request debug log as:

   "secretEnvironmentVariables": [
     ...
     "projectId": "projects/redacted",
     ...
    ]

This means that data.google_project.project.project_id is resolving as projects/redacted and already includes a projects/ prefix. I'm not suggesting that you add the projects/ prefix because according to this logic:

    dynamic "secret_environment_variables" {
        ...
        project_id = "projects/${data.google_project.project.project_id}"
        ...
    }

would result in:

   "secretEnvironmentVariables": [
     ...
     "projectId": "projects/projects/redacted",
     ...
    ]

Do you notice how the projects/ prefix is now doubled as projects/projects/?

Unless this is a property of how your logs got redacted It seems that data.google_project.project.project_id evaluates to projects/YOUR_PROJECT_ID when what you really need is just YOUR_PROJECT_ID. In other words, your debug logs should show:

   "secretEnvironmentVariables": [
     ...
     "projectId": "redacted",
     ...
    ]

assertnotnull commented 10 months ago

I did understand you. It's what I said I tried with data.google_project.project.project_id but it still gave me the same error saying secrets annotation should have the format ...

assertnotnull commented 9 months ago

Any update on this? Also I don't want this to be closed for inactivity.

jakerachleff commented 2 months ago

Not sure it's useful to you @assertnotnull , but I ran into the same issue, and my problem was that I was passing a full ID (e.g. projects/{ID}/secrets/{short_name}) when I should have just been passing in {short_name} to the secret argument.

It was confusing, because other TF modules expected that full ID for my secret.

hashicorp / terraform-provider-google