Error when destroying doppler_config

[alacroix]

Hey,

I use Doppler with Terraform to create temporary environments where developpers can override a base config. So, I'll apply/destroy on a regular basis.

Everything works well when creating the resources, but when I destroy, I get the following error on doppler_config resource:

(I renamed resources and simplified the destroy plan)

  # doppler_config.example will be destroyed
  - resource "doppler_config" "example" {
      - environment = "dev" -> null
      - id          = "example-project.dev.dev_custom_env" -> null
      - name        = "dev_custom_env" -> null
      - project     = "example-project" -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

doppler_config.example: Destroying... [id=example-project.dev.dev_custom_env]
doppler_config.example: Still destroying... [id=example-project.dev.dev_custom_env, 10s elapsed]
╷
│ Error: Doppler Error: Could not find requested config 'dev_custom_env'
│
│

The thing is, the resource is destroyed, but Terraform yields an error, its state is corrupted (since it "failed" to delete but it did it anyway) and the error still shows up.

More info on my setup:

> terraform --version
Terraform v1.3.2
...
doppler = {
  source  = "DopplerHQ/doppler"
  version = "1.1.6"
}

If you have any ideas where this can come from, I'm interested. Thanks!

[nmanoogian]

Thanks for reaching out, @alacroix! I just tried reproducing this locally but I wasn't able to make it happen. Could you provide a full example that I could try out?

Whenever we see deletion failures, the first thing that comes to mind is resource ownership. When you delete an environment in Doppler, all configs under it automatically get deleted. For this reason, it's always a good idea to reference resources with reference values, rather than directly by name. For example,

# ✅ Referencing other Terraform resources
resource "doppler_config" "payments-api-dev-temp" {
  project     = doppler_project.payments-api.name
  environment = doppler_environment.payments-api-dev.slug
  name        = "dev_temp"
}

# ❌ Referencing directly by name
resource "doppler_config" "payments-api-dev-temp" {
  project     = "payments-api"
  environment = "dev"
  name        = "dev_temp"
}

[alacroix]

Thanks for your reply @nmanoogian

To answer your concerns, I've just created/deleted a branch config of an existing base config (dev). Since Doppler Terraform provider doesn't have Data Sources yet, I can't reference other Doppler resources. The project and environments have been created via Doppler dashboard. (My use-case here is only for temporary environment so I don't want to destroy all my project.) I plan to use Terraform for all Doppler resources but still not complete yet

So I have the following:

• Project: example-project
• Environment: Development (slug: dev)
• Config: Branch config named dev_test.

I tried to reproduce in the smallest example I can and even with only one doppler_config resource, I still have the problem:

main.tf

terraform {
  required_version = ">= 0.15.0"

  required_providers {
    doppler = {
      source  = "DopplerHQ/doppler"
      version = "1.1.6"
    }
  }
}

resource "doppler_config" "example" {
  project     = "example-project"
  environment = "dev"
  name        = "dev_test"
}

terraform apply

❯ terragrunt apply

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # doppler_config.example will be created
  + resource "doppler_config" "example" {
      + environment = "dev"
      + id          = (known after apply)
      + name        = "dev_test"
      + project     = "example-project"
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

doppler_config.example: Creating...
doppler_config.example: Creation complete after 2s [id=example-project.dev.dev_test]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

terraform destroy

doppler_config.example: Refreshing state... [id=example-project.dev.dev_test]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # doppler_config.example will be destroyed
  - resource "doppler_config" "example" {
      - environment = "dev" -> null
      - id          = "example-project.dev.dev_test" -> null
      - name        = "dev_test" -> null
      - project     = "example-project" -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

doppler_config.example: Destroying... [id=example-project.dev.dev_test]
doppler_config.example: Still destroying... [id=example-project.dev.dev_test, 10s elapsed]
╷
│ Error: Doppler Error: Could not find requested config 'dev_test'
│
│
╵
ERRO[0016] 1 error occurred:
    * exit status 1

Thanks for your time!

[nmanoogian]

To answer your concerns, I've just created/deleted a branch config of an existing base config (dev). Since Doppler Terraform provider doesn't have Data Sources yet, I can't reference other Doppler resources. The project and environments have been created via Doppler dashboard. (My use-case here is only for temporary environment so I don't want to destroy all my project.) I plan to use Terraform for all Doppler resources but still not complete yet

Ah, I see -- that makes sense! In that case, referencing the project and environment directly in your resources makes sense.

One thing that's jumping out to me in your logs is that it's taking over 10 seconds to delete the config. The provider has a 10 second timeout for requests and it will automatically retry requests which time out. I bet what's happening here is that the first request is successful but the provider abandons it before the response comes back from the Doppler API. The provider re-attempts the request but it fails because the original request was indeed successful.

Two follow-up questions for you:

• Is this failure consistent? Or does it only happen during some deletions?
• When the failure occurs, do the activity logs (in the Doppler dashboard) show that the Terraform provider did actually delete the config?

[alacroix]

Indeed, the 10s timeout makes sense in our case. For the follow-up questions:

• Failure consistent on all my tries
• And yes, on the Activity Log, I do have the line stating that the config has been deleted.

EDIT: Since we can't specify custom timeouts yet, I've tried on a "lighter" Doppler project (~60 secrets) to see if it changes anything and if I can delete the config under 10s. So yeah, I don't have the issue here, so I guess more secrets => more time to delete => timeout in our case. For reference, our base project where I have the original issue has ~200 secrets (I know it's a lot, working on it to merge most of the secrets into objects)

[nmanoogian]

Awesome, thanks for that context! I'll do some testing with large root configs on our end to see if I can track down that slowness.

[nmanoogian]

I'm able to reproduce that slowness when deleting branch configs with ~200 secrets. It shouldn't be taking nearly that long. I've brought this up with the team and we're working on a resolution.

Thanks again for bearing with us and sorry for the trouble!

[alacroix]

Oh that's great news!

No worries and keep me in touch when the problem is solved 🙂

Thanks!

[nmanoogian]

Hey @alacroix, thanks for bearing with me! We're still working on improving the performance of our deletion of large configs, but in the meantime we've increased the request timeout of the provider from 10sec to 30sec. Can you update to terraform-provider-doppler v1.1.7, try deleting one of your ephemeral configs, and let me know if you run into any errors?

[alacroix]

Hey @nmanoogian,

Just updated to v1.1.7, seems to work like a charm!

We can close this issue now.

Thanks!

[nmanoogian]

Awesome, thanks for following up! We're still working behind-the-scenes to improve the performance of that API endpoint but I'm glad we could unblock you in the meantime.