google_cloud_run_v2_service does not register creation in tfstate if initial container boot fails

igorwwwwwwwwwwwwwwwwwwww commented 9 months ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

1.5.3

Affected Resource(s)

google_cloud_run_v2_service

Terraform Configuration Files

resource "google_cloud_run_v2_service" "issue_16765" {
  name     = "issue-16765"
  location = "us-east1"
  ingress  = "INGRESS_TRAFFIC_ALL"

  template {
    containers {
      image = "broken"
    }
  }
}

Debug Output

On initial run:

│ Error: Error waiting to create Service: Error waiting for Creating Service: Error code 13, message: Revision 'redacted-bfda' is not ready and cannot serve traffic. The user-provided container failed to start and listen on the port defined provided by the PORT=8080 environment variable. Logs for this revision might contain more information.
│ 
│ Logs URL: https://console.cloud.google.com/logs/viewer?project=redacted&resource=cloud_run_revision/service_name/redacted/revision_name/redacted-bfda&advancedFilter=resource.type%3D%22cloud_run_revision%22%0Aresource.labels.service_name%3D%22redacted%22 
│ For more troubleshooting guidance, see https://cloud.google.com/run/docs/troubleshooting#container-failed-to-start
│ 
│   with google_cloud_run_v2_service.runway_service,
│   on [main.tf](http://main.tf/) line 84, in resource "google_cloud_run_v2_service" "runway_service":
│   84: resource "google_cloud_run_v2_service" "runway_service" {

This still creates the service on the google side however.

On second run:

│ Error: Error creating Service: googleapi: Error 409: Resource 'redacted' already exists.
│ 
│   with google_cloud_run_v2_service.runway_service,
│   on [main.tf](http://main.tf/) line 84, in resource "google_cloud_run_v2_service" "runway_service":
│   84: resource "google_cloud_run_v2_service" "runway_service" {

Full debug logs: https://gist.github.com/igorwwwwwwwwwwwwwwwwwwww/b59b1aefb22abdd89d03e4179dc4b5d1.

Panic Output

n/a

Expected Behavior

The service is created in GCP, and this resource creation is reflected within the tfstate.

Actual Behavior

The service is created in GCP, but the tfstate does not contain the resource.

Steps to Reproduce

terraform apply

Important Factoids

Not AFAICT.

References

Downstream issue

b/315953949

edwardmedia commented 9 months ago

@igorwwwwwwwwwwwwwwwwwwww both errors appear to be from the api, but no clue what went wrong. Do you have the config that can be used to repro the issue? Also could you share the longer debug log that contains the api requests and responses?

igorwwwwwwwwwwwwwwwwwwww commented 9 months ago

@edwardmedia FYI, the container failing to boot is expected -- it's an application issue. I'm happy to provide a small reproducer though in case that helps.

igorwwwwwwwwwwwwwwwwwwww commented 9 months ago

Reproducer:

resource "google_cloud_run_v2_service" "issue_16765" {
  name     = "issue-16765"
  location = "us-east1"
  ingress  = "INGRESS_TRAFFIC_ALL"

  template {
    containers {
      image = "broken"
    }
  }
}

The container fails to boot because the image is invalid / does not exist in the docker registry.

Debug logs for the two terraform apply runs: https://gist.github.com/igorwwwwwwwwwwwwwwwwwwww/b59b1aefb22abdd89d03e4179dc4b5d1.

igorwwwwwwwwwwwwwwwwwwww commented 9 months ago

Looks like there's a couple other similar reports, possible duplicates of this:

yanweiguo commented 3 months ago

I think this should be fixed by https://github.com/GoogleCloudPlatform/magic-modules/pull/10298

hashicorp / terraform-provider-google