Open asaf-erlich opened 5 years ago
Would you be open to a contribution to add this feature?
@djrodgerspryor I would be open to it in perhaps a few months. My wife and I have a 2 month old and I just don't have time available outside of work to do something like this right now. But I think it's an awesome feature and someone should really do it. I'll try to keep this in mind when I feel like I have the time.
@asaf-erlich do you have any progess on this?
No I apologize. The pandemic + having a baby in the last year is making the feasibility of my working on this extremely unlikely.
@asaf-erlich we're having this issue as well, and I'm happy to contribute to this if it's not yet started by you.
Hello @billcchung, I never made any progress on this issue. Feel free to make any contribution if you have the bandwidth to do so.
hi @asaf-erlich I've opened a PR https://github.com/hashicorp/terraform/pull/32287 for this, would you mind taking a look?
Hello @billcchung , I can try to take a look either this week or next week and do my best to provide feedback. I would like to point out that I'm not a maintainer of terraform nor have I ever written code for this code base. Just saw that 2 other people already responded on the PR so hopefully that is not an issue. Either way I appreciate you submitting this PR. Thank you.
Has this ever been resolved ?
Unfortunately the draft pr was never finished.
Thanks ☺️
Current Terraform Version
Use-cases
Terraform creates aws resources. Sometimes the job which executes terraform crashes (in our particular use case a Jenkins job might crash during Jenkins master restart, which is a bad architecture design of Jenkins but that's besides the point, the terraform process can crash randomly either way). From that point on the lock in dynamo db will be held forever or until manual action is taken to release the lock.
We have a cleanup job that runs hours later (executes terraform destroy for all the modules) that would normally clean up these aws resources, but it will fail because the lock is held forever (unless manual action is taken).
Attempted Solutions
At the moment every time this fails someone has to manually go into dyanamo db in aws using either the console or cli and delete the locked resource lockID item. Then when the job to clean up resource is executed it works.
Proposal
Dynamo db tables already offer a way to add a row / value with a TTL: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/time-to-live-ttl-how-to.html
It would be very nice if this could be just something that is configured. My suggestion is just to add a value for ttl inside the terraform backend configuration, like dynamodb_lock_ttl. From an implementation point of view if the value is not set for an existing table and it needs to be enabled this could be problematic. But terraform could just throw a warning that it only supports ttl for newly created tables.
References
Potentially this is related: https://github.com/hashicorp/terraform/issues/15429
Let me know if you need more information. Thank you, -Asaf