Resource was not found but does exist on GCE terraform apply

[wint00]

I often get errors like this and the machines failed to get created:

* Error loading disk 'pd-wintoo-app-00': googleapi: Error 404: The resource 'projects/wintoo-staging/zones/us-central1-a/disks/pd-wintoo-app-00' was not found, notFound
* Error loading disk 'pd-wintoo-app-01': googleapi: Error 404: The resource 'projects/wintoo-staging/zones/us-central1-b/disks/pd-wintoo-app-01' was not found, notFound
* Error loading disk 'pd-wintoo-app-02': googleapi: Error 404: The resource 'projects/wintoo-staging/zones/us-central1-c/disks/pd-wintoo-app-02' was not found,

But when I run terraform again, it usually just works. Is that expected and should I ignore or should I be putting more sleeps in order to prevent this? Perhaps I should look something up before allowing the VM to be created?

[catsby]

Hey @wint00 – do you have a configuration that demonstrates this issue in a reproducible manner? This doesn't seem like expected behavior, anything additional you have to help track this down would be much appreciated. If you do, please be sure to omit any secrets

[wint00]

Hi,

I can but I think it's pretty easy to trigger. I haven't tried the simplest case but it certainly happens when trying to create about 20 resources -- if I create a file called: disk.tf and put some definition of a disk and then create a file called instance.tf and reference that disk file the mentioned behavior will occur. I went around it by explicitly adding a 'depends on' section to my instance.tf. However, I assume that since you ask for the disk, it must be created first and there is a lot of mention of how terraform infers dependencies. In this case it did not. If you can't repro, I can send over my broken stuff w/ all the secret stuff removed.

Cheers.

[wint00]

As a followup, when I do a terraform destroy operation my disks that I created separately, but have a depends_on statement spit this out: Error deleting disk: googleapi: Error 404: The resource 'projects/wintoo-staging/zones/us-central1-a/disks/pd-wintoo-00-a-data

The configuration first creates these disks above and then creates hosts that they (happily) attach to. Once I do a destroy, the disks pd-wintoo-00-a-data are destroyed and then as the host is being destroyed it complains the disk doesn't exist. Well of course it doesn't Terraform! You just destroyed them.

Here is my disk.tf

resource "google_compute_disk" "pd-wintoo-00-data" {
  count = "5"
  name = "pd-wintoo-00-${element(split(",", var.os_replicas), count.index)}-data"
  type = "pd-ssd"
  zone = "${var.region}-${element(split(",", var.zones), count.index)}"
  size = "500"
}

resource "google_compute_disk" "pd-wintoo-01-data" {
  count = "5"
  name = "pd-wintoo-01-${element(split(",", var.os_replicas), count.index)}-data"
  type = "pd-ssd"
  zone = "${var.region}-${element(split(",", var.zones), count.index)}"
  size = "500"
}

resource "google_compute_disk" "pd-wintoo-02-data" {
  count = "5"
  name = "pd-wintoo-02-${element(split(",", var.os_replicas), count.index)}-data"
  type = "pd-ssd"
  zone = "${var.region}-${element(split(",", var.zones), count.index)}"
  size = "500"
}

resource "google_compute_disk" "pd-wintoo-03-data" {
  count = "5"
  name = "pd-wintoo-03-${element(split(",", var.os_replicas), count.index)}-data"
  type = "pd-ssd"
  zone = "${var.region}-${element(split(",", var.zones), count.index)}"
  size = "500"
}

And here is my db.tf that creates the hosts this machine will attach to:

resource "google_compute_instance" "wintoo-00" {
  count = "5"
  name = "wintoo-00-${element(split(",", var.os_replicas), count.index)}"
  machine_type = "${var.os_type}"
  zone = "${var.region}-${element(split(",", var.zones), count.index)}"
  tags = ["ubuntu", "wint00", "wintoo"]
  depends_on = ["google_compute_disk.pd-wintoo-00-data"]

  disk {
    image = "${var.os_image}"
    size = "100"
    device_name= "pd-wintoo-00-${element(split(",", var.os_replicas), count.index)}"
  }

  disk {
    disk = "pd-wintoo-00-${element(split(",", var.os_replicas), count.index)}-data"
  }

# Need to explicitly let it know who to connect as, defaults don't work
  connection {
        user = "${var.gce_ssh_user}"
        key_file = "${var.gce_ssh_private_key_file}"
        agent = "true"
  }
  network_interface {
        network = "default"
        access_config {
            // Ephemeral IP
        }
  }

  provisioner "local-exec" {
        command = "./python -c 'from cloud_config import cc_render; print cc_render(\"ubuntu-1404-common.cloud-config\", \"ubuntu-artifact.cloud-config\", \"ubuntu-core-report.cloud-config\", \"wintoo-wint00.cloud-config\")' > wint00.cloud-config"
  }

  metadata {
    wintoo-cluster = "configserver"
    fast-cycle = "wintoo-db"
    health-check-endpoints = "9000"
    user-data = "${file("wint00.cloud-config")}"
    startup-script = "${file("startup/wintoo.sh")}"
  }

  service_account {
    scopes = ["storage-rw"]
  }
}

And of course this repeats for the other 3 types of hosts as well.

[wint00]

One way to fix this is to allow depends on to have interpolations: value cannot contain interpolations that way when I create a depends_on statement it only depends on the exact instance of the resource I'm depending on. Now (if you look at the script above) if I create the compute unit it first creates all the disks for that stanza. That's not necessary and the exception in most cases I think.

[sparkprime]

You can also create a dependency on the disk by referring to an attribute, in this case .name

${google_compute_disk.pd-wintoo-03-data.name}

I am not sure if it is possible to do that using element and split, but if not, that is a core problem.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.