Terraform code fails to update the instance in GCP on sole-tenant

[ghost]

Affected Resource(s)

• google_compute_instance

This issue was originally opened by @imerchant-clgx as hashicorp/terraform#22693. It was migrated here as a result of the provider split. The original body of the issue is below.

---

Terraform Version

Terraform v0.11.13
We tried upgrading terraform to version 12.7 but that did not help at all.

Terraform Configuration Files

scheduling { 
automatic_restart = true 
on_host_maintenance = "MIGRATE" 
preemptible = false 
node_affinities = "${var.st_node_affinity}" 
} 

Debug Output

Crash Output

Expected Behavior

When modifying tfvars file for an instance and doing a terraform apply should modify the instance. I am trying to change serviceaccount or machine-type associated with a GCP VM.

Actual Behavior

The modification fails with error message: Error: Error applying plan:

1 error(s) occurred:

• module.instance.google_compute_instance.vm_instance: 1 error(s) occurred:

• google_compute_instance.vm_instance: Error updating scheduling policy: googleapi: Error 400: Cannot update scheduling for instance with node affinities., badRequest

Terraform does not automatically rollback in the face of errors. Instead, your Terraform state file has been partially updated with any resources that successfully completed. Please address the error above and apply again to incrementally change your infrastructure.

Steps to Reproduce

1. terraform init
2. terraform apply

Additional Context

Our terraform code is working on creating/modifying VMs on a Sole-tenant node group. So the code with node-affinity causes issue.

We upgraded terraform to version 12.7 and tried as well. However, that broke my code completely.

References

[rileykarson]

Hey @imerchant-clgx! Can you share the before and after configs you're using, and debug logs?

[imerchant-clgx]

Hello @rileykarson. I am attaching the code tf files for you here. I have not collected any debug logs, but I will try collect them in next couple of days and I will upload them as well.

[rileykarson]

I took a look at the config, but didn't see anything out of place. I'd be interested in the diff shown in terraform plan + the request sent in your debug logs.

[imerchant-clgx]

@rileykarson - I have created a terraform.log file for your review. Can I request you to provide me with an upload location to share with you which is not public. Thanking you in advance.

[rileykarson]

Sure! You can send it to my GH username @google.com.

[g-awmalik]

@imerchant-clgx - you can also upload it to the google support case you started. Riley has access to that as well.

[g-awmalik]

@imerchant-clgx - also can you share the alternative you have with gcloud that you're trying to build with TF?

[imerchant-clgx]

@awaismalik82 / @rileykarson, I have uploaded the terraform error log file to the Google Case for you review.

[rileykarson]

I repro'ed this locally, and dug into it; this is a case of Terraform [Core] acting incorrectly. It's an occurrence of a longstanding bug where Terraform believes fields that weren't changed have been changed based on their addresses in memory changing, https://github.com/hashicorp/terraform/issues/17411. Terraform sees a diff on the scheduling block that won't go away.

I've attempted a couple fixes, but none have taken- it's a particularly problematic issue because the diff persists even in cases when it shouldn't exist. I've bumped that issue, and I'll look into another mitigation soon.

[danawillow]

@rileykarson I know this was a while back, but do you remember whether this was a permadiff or just the apply-time error? If the latter, could we add extra logic after HasChange to check whether old and new are equivalent?

[rileykarson]

I believe it was a permadiff

[karolgorc]

Is this still an issue?