Error updating apigee organization

[apichick]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.5.7 on darwin_arm64

• provider registry.terraform.io/hashicorp/archive v2.4.0
• provider registry.terraform.io/hashicorp/google v4.83.0
• provider registry.terraform.io/hashicorp/google-beta v4.83.0

Affected Resource(s)

• google_apigee_organization

Terraform Configuration Files

I am updating the authorized_network in an Apigee organization using the following resource

resource "google_apigee_organization" "organization" {
  count                                = var.organization == null ? 0 : 1
  analytics_region                     = var.organization.analytics_region
  project_id                           = var.project_id
  authorized_network                   = var.organization.authorized_network
  billing_type                         = var.organization.billing_type
  runtime_type                         = var.organization.runtime_type
  runtime_database_encryption_key_name = var.organization.database_encryption_key
  retention                            = var.organization.retention
  disable_vpc_peering                  = var.organization.disable_vpc_peering
}

Debug Output

The update works fine in the log file, but after the update it seems to trigger a loop where it tries to hit a URL that is constantly returning a 404 until it exceeds the retries. See below the debug output:

module.apigee_x_prod.module.apigee[0].google_apigee_organization.organization[0]: Modifying... [id=organizations/apigee-x-prod]
2023-09-22T17:12:30.997+0200 [INFO]  Starting apply for module.apigee_x_prod.module.apigee[0].google_apigee_organization.organization[0]
2023-09-22T17:12:30.997+0200 [DEBUG] module.apigee_x_prod.module.apigee[0].google_apigee_organization.organization[0]: applying the planned Update change
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:30 [DEBUG] Updating Organization "organizations/apigee-x-prod": map[string]interface {}{"analyticsRegion":"europe-west4", "authorizedNetwork":"projects/networking-prod/global/networks/hub-vpc", "billingType":"SUBSCRIPTION", "name":"apigee-x-prod", "properties":map[string]interface {}{"property":[]interface {}{map[string]interface {}{"name":"features.hybrid.enabled", "value":"true"}, map[string]interface {}{"name":"features.mart.connect.enabled", "value":"true"}}}, "runtimeDatabaseEncryptionKeyName":"projects/apigee-x-prod/locations/global/keyRings/apigee/cryptoKeys/database-key", "runtimeType":"CLOUD"}
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:30 [DEBUG] Waiting for state to become: [success]
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:30 [DEBUG] Retry Transport: starting RoundTrip retry loop
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:30 [DEBUG] Retry Transport: request attempt 0
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:30 [DEBUG] Google API Request Details:
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: ---[ REQUEST ]---------------------------------------
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: PUT /v1/organizations/apigee-x-prod?alt=json HTTP/1.1
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Host: apigee.googleapis.com
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: User-Agent: Terraform/1.5.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google/4.83.0
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Content-Length: 459
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Content-Type: application/json
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Accept-Encoding: gzip
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: {
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:  "analyticsRegion": "europe-west4",
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:  "authorizedNetwork": "projects/networking-prod/global/networks/hub-vpc",
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:  "billingType": "SUBSCRIPTION",
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:  "name": "apigee-x-prod",
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:  "properties": {
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "property": [
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:    {
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:     "name": "features.hybrid.enabled",
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:     "value": "true"
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:    },
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:    {
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:     "name": "features.mart.connect.enabled",
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:     "value": "true"
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:    }
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   ]
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:  },
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:  "runtimeDatabaseEncryptionKeyName": "projects/apigee-x-prod/locations/global/keyRings/apigee/cryptoKeys/database-key",
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:  "runtimeType": "CLOUD"
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: }
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:
2023-09-22T17:12:30.999+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: -----------------------------------------------------
2023-09-22T17:12:32.107+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:32 [DEBUG] Google API Response Details:
2023-09-22T17:12:32.107+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: ---[ RESPONSE ]--------------------------------------
2023-09-22T17:12:32.107+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: HTTP/2.0 200 OK
2023-09-22T17:12:32.107+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2023-09-22T17:12:32.107+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Cache-Control: private
2023-09-22T17:12:32.107+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Content-Type: application/json; charset=UTF-8
2023-09-22T17:12:32.107+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Date: Fri, 22 Sep 2023 15:12:32 GMT
2023-09-22T17:12:32.107+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Server: ESF
2023-09-22T17:12:32.107+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Vary: Origin
2023-09-22T17:12:32.110+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Vary: X-Origin
2023-09-22T17:12:32.110+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Vary: Referer
2023-09-22T17:12:32.110+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Content-Type-Options: nosniff
2023-09-22T17:12:32.110+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Frame-Options: SAMEORIGIN
2023-09-22T17:12:32.110+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Backends: [::1]:26092,/bns/ue/borg/ue/bns/edge-gaambo/prod.edge-gaambo/2,/bns/we/borg/we/bns/traffic-prod/shared-layer2-gfe/70,/bns/we/borg/we/bns/traffic-prod/shared-layer2-gfe/70,/bns/lcmada/borg/lcmada/bns/blue-layer1-gfe-prod-edge/prod.blue-layer1-gfe.mad41s07/40
2023-09-22T17:12:32.110+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Dos-Service-Trace: main:edge-gaambo-api-prod,main:GLOBAL_all_non_cloud,main:shared-layer2-gfe
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Esf-Cloud-Client-Params: backend_service_name: "apigee.googleapis.com" backend_fully_qualified_method: "google.cloud.apigee.v1.OrganizationService.UpdateOrganization"
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gfe-Handshake-Trace: GFE: /bns/lcmada/borg/lcmada/bns/blue-layer1-gfe-prod-edge/prod.blue-layer1-gfe.mad41s07/40,Mentat oracle: [2002:a05:601a:2203::]:9838
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gfe-Request-Trace: acmadp18:443,wemt10:9805,/bns/ue/borg/ue/bns/edge-gaambo/prod.edge-gaambo/2,wemt10:9805,acmadp18:443
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gfe-Response-Body-Transformations: chunked
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gfe-Response-Code-Details-Trace: response_code_set_by_backend,response_code_set_by_backend
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gfe-Service-Trace: edge-gaambo-api-prod,restricted-shared-layer2-grpc-aggregate
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gfe-Version: 2.843.1
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gslb-Service: edge-gaambo-api-prod
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Netmon-Label: /bns/ue/borg/ue/bns/edge-gaambo/prod.edge-gaambo/2
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Security-Signals: FRAMEWORK=ONE_PLATFORM
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Security-Signals: FRAMEWORK=HTTPSERVER2,BUILD=GOOGLE3,BUILD_DEBUG=cl:557990377
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Service: edge-gaambo-api-prod,restricted-shared-layer2-grpc-aggregate
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Session-Info: CIyNwrTBDhoEGBAoCygBOjUSFWFwaWdlZS5nb29nbGVhcGlzLmNvbRiO_tm30xMiFTExNTIyNDUyMzE0NjkwMjQ2MDM3MkoYQhZOT1RfQV9QRVJTSVNURU5UX1RPS0VO
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Shellfish-Status: CA0gBEBG
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Xss-Protection: 0
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: {
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "name": "apigee-x-prod",
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "createdAt": "1692906362643",
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "lastModifiedAt": "1695395552067",
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "environments": [
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: "00"
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   ],
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "properties": {
2023-09-22T17:12:32.112+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:     "property": [
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:       {
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:         "name": "features.mart.connect.enabled",
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:         "value": "true"
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:       },
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:       {
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:         "name": "features.hybrid.enabled",
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:         "value": "true"
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:       }
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:     ]
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   },
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "analyticsRegion": "europe-west4",
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "authorizedNetwork": "projects/networking-prod/global/networks/hub-vpc",
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "runtimeType": "CLOUD",
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "subscriptionType": "PAID",
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "caCertificate": "***",
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "runtimeDatabaseEncryptionKeyName": "projects/apigee-x-prod/locations/global/keyRings/apigee/cryptoKeys/database-key",
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "state": "ACTIVE",
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   "billingType": "SUBSCRIPTION"
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: }
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: -----------------------------------------------------
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:32 [DEBUG] Retry Transport: Stopping retries, last request was successful
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:32 [DEBUG] Retry Transport: Returning after 1 attempts
2023-09-22T17:12:32.113+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:32 [DEBUG] Finished updating Organization "organizations/apigee-x-prod": map[string]interface {}{"analyticsRegion":"europe-west4", "authorizedNetwork":"projects/networking-prod/global/networks/hub-vpc", "billingType":"SUBSCRIPTION", "caCertificate":"***", "createdAt":"1692906362643", "environments":[]interface {}{"00"}, "lastModifiedAt":"1695395552067", "name":"apigee-x-prod", "properties":map[string]interface {}{"property":[]interface {}{map[string]interface {}{"name":"features.mart.connect.enabled", "value":"true"}, map[string]interface {}{"name":"features.hybrid.enabled", "value":"true"}}}, "runtimeDatabaseEncryptionKeyName":"projects/apigee-x-prod/locations/global/keyRings/apigee/cryptoKeys/database-key", "runtimeType":"CLOUD", "state":"ACTIVE", "subscriptionType":"PAID"}
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:32 [DEBUG] Waiting for state to become: [done: true]
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:32 [DEBUG] Waiting for state to become: [success]
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:32 [DEBUG] Retry Transport: starting RoundTrip retry loop
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:32 [DEBUG] Retry Transport: request attempt 0
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:32 [DEBUG] Google API Request Details:
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: ---[ REQUEST ]---------------------------------------
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: GET /v1/apigee-x-prod?alt=json HTTP/1.1
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Host: apigee.googleapis.com
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: User-Agent: Terraform/1.5.7 (+https://www.terraform.io) Terraform-Plugin-SDK/2.10.1 terraform-provider-google/4.83.0
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Content-Type: application/json
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Accept-Encoding: gzip
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:
2023-09-22T17:12:32.114+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: -----------------------------------------------------
2023-09-22T17:12:32.289+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:32 [DEBUG] Google API Response Details:
2023-09-22T17:12:32.289+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: ---[ RESPONSE ]--------------------------------------
2023-09-22T17:12:32.289+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: HTTP/2.0 404 Not Found
2023-09-22T17:12:32.289+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Content-Length: 1597
2023-09-22T17:12:32.289+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2023-09-22T17:12:32.289+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Content-Type: text/html; charset=UTF-8
2023-09-22T17:12:32.289+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Date: Fri, 22 Sep 2023 15:12:32 GMT
2023-09-22T17:12:32.289+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Server: ESF
2023-09-22T17:12:32.289+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Content-Type-Options: nosniff
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Frame-Options: SAMEORIGIN
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Backends: /bns/ue/borg/ue/bns/edge-gaambo/prod.edge-gaambo/4,/bns/wm/borg/wm/bns/traffic-prod/shared-layer2-gfe/45,/bns/wm/borg/wm/bns/traffic-prod/shared-layer2-gfe/45,/bns/lcmada/borg/lcmada/bns/blue-layer1-gfe-prod-edge/prod.blue-layer1-gfe.mad41s07/40
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Dos-Service-Trace: main:edge-gaambo-api-prod,main:GLOBAL_all_non_cloud,main:shared-layer2-gfe
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gfe-Handshake-Trace: GFE: /bns/lcmada/borg/lcmada/bns/blue-layer1-gfe-prod-edge/prod.blue-layer1-gfe.mad41s07/40,Mentat oracle: [2002:a05:601a:2203::]:9838
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gfe-Request-Trace: acmadp18:443,wmnv7:9805,/bns/ue/borg/ue/bns/edge-gaambo/prod.edge-gaambo/4,wmnv7:9805,acmadp18:443
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gfe-Response-Code-Details-Trace: response_code_set_by_backend,response_code_set_by_backend
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gfe-Service-Trace: edge-gaambo-api-prod,restricted-shared-layer2-grpc-aggregate
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gfe-Version: 2.843.1
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Gslb-Service: edge-gaambo-api-prod
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Netmon-Label: /bns/ue/borg/ue/bns/edge-gaambo/prod.edge-gaambo/4
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Security-Signals: FRAMEWORK=HTTPSERVER2,BUILD=GOOGLE3,BUILD_DEBUG=cl:557990377
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Service: edge-gaambo-api-prod,restricted-shared-layer2-grpc-aggregate
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Shellfish-Status: CA0gBEBG
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Xss-Protection: 0
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: <!DOCTYPE html>
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: <html lang=en>
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   <meta charset=utf-8>
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   <title>Error 404 (Not Found)!!1</title>
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   <style>
2023-09-22T17:12:32.290+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:     *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   </style>
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   <a href=//www.google.com/><span id=logo aria-label=Google></span></a>
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   <p><b>404.</b> <ins>That’s an error.</ins>
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:   <p>The requested URL <code>/v1/apigee-x-prod?alt=json</code> was not found on this server.  <ins>That’s all we know.</ins>
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5:
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: -----------------------------------------------------
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: 2023/09/22 17:12:32 [DEBUG] Retry Transport: Stopping retries, last request failed with non-retryable error: googleapi: got HTTP response code 404 with body: HTTP/2.0 404 Not Found
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Content-Length: 1597
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Content-Type: text/html; charset=UTF-8
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Date: Fri, 22 Sep 2023 15:12:32 GMT
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: Server: ESF
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Content-Type-Options: nosniff
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Frame-Options: SAMEORIGIN
2023-09-22T17:12:32.291+0200 [DEBUG] provider.terraform-provider-google_v4.83.0_x5: X-Google-Backends: /bns/ue/borg/ue/bns/edge-gaambo/prod.edge-gaambo/4,/bns/wm/borg/wm/bns/traffic-prod/shared-layer2-gfe/45,/bns/wm/borg/wm/bns/traffic-prod/shared-layer2-gfe/45,/bns/lcmada/borg/lcmada/bns/blue-layer1-gfe-prod-edge/prod.blue-layer1-gfe.mad41s07/40

Because of that the apply ends erroring out the following:

│ Error: Error waiting for Updating Organization: couldn't find resource (21 retries)
│
│   with module.apigee_x_prod.module.apigee[0].google_apigee_organization.organization[0],
│   on .terraform/modules/apigee_x_prod.apigee/modules/apigee/main.tf line 22, in resource "google_apigee_organization" "organization":
│   22: resource "google_apigee_organization" "organization" {
│

It looks to me that the URL that is being built to hit in this wait condition here is not ok

https://github.com/hashicorp/terraform-provider-google/blob/main/google/services/apigee/resource_apigee_organization.go#L482

http://apigee.googleapis.com/v1/{project_id} is not a vailid url. Check here the Apigee API reference.

Could someone have a look at this and confirm that it is actually a bug?

Thanks.

b/301736369

[dallasanta]

I'm getting exactly the same issue with the same underlying problem. Would be great to have it fixed.