hetznercloud / terraform-provider-hcloud

Terraform Hetzner Cloud provider
https://registry.terraform.io/providers/hetznercloud/hcloud/latest
Mozilla Public License 2.0
513 stars 74 forks source link

[Bug]: Machine creation takes a long time #503

Closed whoracle closed 1 year ago

whoracle commented 2 years ago

What happened?

When creating a new machine via terraform, the process takes upwards of 15 minutes to complete. Via the WebUI the machine deployment is much faster.

We're using a snapshot of another machine to create the VM in question.

What did you expect to happen?

Machine creation should not take significantly longer via terraform than via the WebUI

Please provide a minimal working example

resource "hcloud_server" "this" { name = test.whatever.com image = "snapshot_numeric_id" server_type = "cx11" location = "nbg1 backups = false

labels = { managed_by = "terraform" }

lifecycle { ignore_changes = [ image, ] } }

LKaemmerling commented 2 years ago

Hey @whoracle,

can you still reproduce it? In general there is no difference between the Call from the Cloud Console (WebUI) and terraform. It could be the case that your snapshot was stored in Helsinki and that your snapshot is quite big, of course, because of the distance between HEL & NBG and the size of the snapshot it can happen that the server creation takes a bit longer.

whoracle commented 2 years ago

Thanks for the feedback - the Snapshot in question is indeed located in HEL. I'll rebuild it tonight and will get back to you tomorrow once I have it in NBG and see if that changes anything.

whoracle commented 2 years ago

Been a while since I got around to this:

We've moved the snapshot that we use to spawn new VMs from HEL to NBG. Still seeing 15+ minutes to completion for a module consisting of hcloud_vm and hcloud_rdns, no matter where we locate the VMs. Spawning the same via WebUI takes somewhere in the 2 minute range.

apricote commented 2 years ago

Hi @whoracle,

it would be great if you could send me debug logs for this, so I can investigate what is happening here.

To generate the logs you can use the TF_LOG=DEBUG environment variable:

TF_LOG=DEBUG TF_LOG_PATH=slow-server-create.log terraform apply

Please make sure that the log actually contains the create server call. It should contain this string POST /v1/servers.

Afterwards, you can send it to me by email: julian.toelle<at>hetzner-cloud.de

whoracle commented 2 years ago

Hi @apricote

Just sent you the mail. Log is ~22MB (refresh and module stuff for existing infrastructure). Crossposting the other information here for posterity.

Used the same image as usual, created a test vm (via apply -target):

resource "hcloud_server" "testvm" {
  name        = "testvm"
  image       = 89557991
  server_type = "cx21"
  location    = "nbg1"
  backups     = false
}

terraform apply started at 15:29 got a mail from hetzner about successful creation at 15:45 run finished according to my terminal at 15:46

Deletion of the VM afterwards took less than a minute.

Thanks for looking into this.

whoracle commented 1 year ago

Oh dear god. Fixed it after a VERY helpful pointer by @apricote :

In my provider config I had poll_interval set to "500s" instead of "500ms" (which is also the default of the provider).

Head, meet desk.

Closing this in shame.