terracurl Destroy Operation Uses Stale ID Token from State File

[amuware]

Describe the issue We are encountering an issue when using the terracurl module to interact with our API hosted on GCP Cloud Run. Our Terraform module dynamically generates a GCP ID token using the google_service_account_id_token data source to authenticate API requests.

The terracurl resource works correctly when creating (PUT request) resources, using the freshly generated ID token. However, during the destroy operation (DELETE request), terracurl uses the ID token stored in the Terraform state file, which might have expired, leading to a 401 Unauthorized error from Cloud Run.

Steps to Reproduce

1. Generate a dynamic ID token: Use google_service_account_id_token data source to generate a new GCP ID token each time the module is applied.

2. Set the token in the terracurl headers: Use the generated ID token in the headers and destroy_headers of the terracurl_request resource.

3. Run terraform apply: The resources are created successfully using the correct ID token.

4. Wait for the token to expire: After approximately 1 hour, the generated ID token expires.

5. Run terraform destroy: A new ID token is generated, but terracurl does not use it. Instead, it uses the expired token from the Terraform state, resulting in a 401 error during the destroy operation.

Expected Behavior

• Token Refresh During Destroy: When the terraform destroy command is executed, terracurl should regenerate or use the freshly generated ID token from the google_service_account_id_token data source, rather than the stale token stored in the Terraform state file.

Current Behavior

• Stale Token Usage: terracurl references the ID token from the state file during the destroy operation, which often leads to a 401 Unauthorized error if the token has expired.

data "google_service_account" "sa_id" {
  account_id = "sa-gcp"
  project    = var.project
}

data "google_service_account_id_token" "sa_gcp" {
  target_service_account = data.google_service_account.sa_id.email
  include_email          = true
  target_audience        = "https://test-dev-xcgee4a-us.a.run.app/"
}

resource "terracurl_request" "a_record" {
  for_each = length(var.public_type) > 0 ? { for rec in local.full_records : rec.sub => rec } : {}

  name    = "${var.env}_${each.key}_public_record"
  method  = "PUT"
  url     = "https://test-dev-xcgee4a-us.a.run.app/XXX/rec"

  headers = {
    "X-Serverless-Authorization" = "Bearer ${data.google_service_account_id_token.sa_gcp.id_token}"
    "Content-Type"               = "application/json"
  }

  request_body = jsonencode({
    ...
  })

  response_codes = [200]
  timeout        = 120

  destroy_timeout = 120
  destroy_url     = "https://test-dev-xcgee4a-us.a.run.app/XXX/rec"
  destroy_method  = "DELETE"
  destroy_headers = {
    "X-Serverless-Authorization" = "Bearer ${data.google_service_account_id_token.sa_gcp.id_token}"
    "Content-Type"               = "application/json"
  }
  destroy_request_body = jsonencode({
    ...
  })

  destroy_response_codes = [200, 401, 500]

  depends_on = [data.google_service_account_id_token.sa_gcp]
}

• The following is a sippet from the terraform desroy:

  
  # module.test.terracurl_request.a_record["test"] will be destroyed
  - resource "terracurl_request" "public_a_records" {
    - destroy_headers        = {
        - "Content-Type"               = "application/json"
        - "X-Serverless-Authorization"        = "Bearer mEPu4ld9wgFYsrDoO2xLgzn7rt0EnQ" 

• headers = {
  • "Content-Type" = "application/json"
  • "X-Serverless-Authorization" = "Bearer mEPu4ld9wgFYsrDoO2xLgzn7rt0EnQ" --> this is an old token not the one created dynamically with google_service_account_id_token

Further Explanation

• The key issue is that Terraform's state file retains the ID token generated at the time of resource creation. During the destroy operation, terracurl uses this outdated token rather than regenerating a fresh one. This leads to authorization errors when the token has expired.

• A potential solution within the terracurl module could involve re-evaluating the ID token before any destroy operation, ensuring that the destroy_headers field is populated with a valid, current token.

• Alternatively, introducing a mechanism within the module to invalidate the stored token and force a refresh during destroy operations might solve this issue.

Thank you for your attention to this matter, and I look forward to your guidance or an update to the terracurl module that addresses this issue.

[amuware]

Hello @devops-rob, any ideas regarding this issue ? Thanks