[ISSUE] Issue with `dashboard` resource (Provider produced inconsistent final plan)

[davidvega-db]

Configuration

Doc provided in the steps

Expected Behavior

Applies must be consistent

Actual Behavior

The issue occurs when you run another terraform apply after the first apply. You get the error: 14:30:58 2024-10-02T19:30:58.340Z [DEBUG] [aws-sdk-go] {} 14:30:58
14:30:58 Error: Provider produced inconsistent final plan 14:30:58
14:30:58 When expanding the plan for 14:30:58 databricks_dashboard.your_dashboard to include new values 14:30:58 learned so far during apply, provider 14:30:58 "registry.terraform.io/databricks/databricks" produced an invalid new value 14:30:58 for .serialized_dashboard: was null, but now 14:30:58 cty.StringVal

Steps to Reproduce

Create a Databricks dashboard using the “serialized_dashboard” attribute via terraform using the instructions in the following link: https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/dashboard switch over to using the “file_path” attribute. To do that, run a terraform destroy (since the docs above say you have to destroy if you switch between those attributes). After the terraform destroy, run a terraform apply to recreate the infrastructure. That works fine. The issue occurs when you run another terraform apply after the first apply. You get the error.

Terraform and provider versions

1.52.0

Is it a regression?

Unknown

Debug Output

14:30:58 2024-10-02T19:30:58.340Z [DEBUG] [aws-sdk-go] {} 14:30:58
14:30:58 Error: Provider produced inconsistent final plan 14:30:58
14:30:58 When expanding the plan for 14:30:58 databricks_dashboard.your_dashboard to include new values 14:30:58 learned so far during apply, provider 14:30:58 "registry.terraform.io/databricks/databricks" produced an invalid new value 14:30:58 for .serialized_dashboard: was null, but now 14:30:58 cty.StringVal

[pietern]

Thanks for reporting.

I can reproduce this if I remove the dynamically generated file (referred to as file_path) between the calls to apply.

But the apply call after the failing one succeeds because the dynamically generated .lvdash.json file exists during planning. The error happens because the file doesn't exist during planning, so TF expects the contents to be null. Then, during the apply phase, the file exists, is loaded, and the contents turn out not to be null.

Not sure yet if we can address this issue in the provider.

[pietern]

This is a more fundamental issue than I realized initially: TF cannot accurately plan whether ot not a dashboard resource should be updated or not if the file with the dashboard configuration doesn't exist. The file has to exist during planning. To make this happen in your setup you can either first run a targeted apply of whatever it is that creates the file, followed by an apply of everything, or you could break it out into a separate process/step/Makefile.

Closing the issue because I don't think we can do anything about this.

[pietern]

Minimal reproduction of the issue: https://gist.github.com/pietern/676d0a1f256ffb6a4ca8654bb8ed21e3