Resources not cleaned up from test, incurring ongoing charges in Aiven account

pulumi / pulumi-aiven

An Aiven Pulumi resource package, providing multi-language access to Aiven

Apache License 2.0

12 stars 2 forks source link

Resources not cleaned up from test, incurring ongoing charges in Aiven account #235

Open AaronFriel opened 1 year ago

AaronFriel commented 1 year ago

[x] Suspend tests
[x] Document account (likely bot@pulumi.com) used: answer - in engineering team 1password vault.
[x] Clean up outstanding resources in account
[ ] Follow up with @isaac-pulumi on approach to re-enable tests or future steps.

AaronFriel commented 1 year ago

Removing P1 and keeping this as a tracking issue.

guineveresaenger commented 1 year ago

I recommend a cleaner-upper, which has been used as a solution in other providers: https://github.com/pulumi/civo-account-cleanup

t0yv0 commented 1 year ago

Just merged something that enables tests back, should they still be skipped?

AaronFriel commented 1 year ago

We should keep this open for a while, I'll track our Aiven portal and cron workflow to see if we're leaving resources alive. There's an upstream issue enabling us to tweak timeouts via a fork + patch or ldflags:

https://github.com/aiven/terraform-provider-aiven/issues/1054

t0yv0 commented 1 year ago

Also https://github.com/pulumi/ci-mgmt/pull/379 perhaps relevant.

I don't mind keeping this open, just FYI more tests are running now in Aiven that were skipped before today. Just in case this causes further problems.

AaronFriel commented 1 year ago

In the case of this issue, the cleanup failed due to provider timeouts. The physical resource creation succeeded async after the provider timed out, so the logical resource create call failed. The state file thus had no resources to clean up when programtest ran pulumi destroy.