Bulk state modifications

nl-brett-stime commented 2 years ago

Current Terraform Version

Terraform v1.0.11

...

Use-cases

It'd be great if we could:

Lock shared TF state once
Issue a bunch of [batched] commands to modify the state
Release the state lock

...instead of having to re-acquire and re-release the lock for each mutation/statement.

E.g., instead of

% terraform state mv module.some.module.nested1.resource_type.old_name module.some.module.nested1.resource_type.new_name
  Acquiring lock...
  Moved succeeded!
  Releasing lock...
% terraform state mv module.some.module.nested2.resource_type.old_name module.some.module.nested2.resource_type.new_name
  Acquiring lock...
  Moved succeeded!
  Releasing lock...
% terraform state mv module.other.module.nested1.resource_type.old_name module.other.module.nested1.resource_type.new_name
  Acquiring lock...
  Moved succeeded!
  Releasing lock...
% terraform state mv module.other.module.nested2.resource_type.old_name module.other.module.nested2.resource_type.new_name
  Acquiring lock...
  Moved succeeded!
  Releasing lock...

...it'd be nice if we could e.g.,:

% terraform state batch begin \
mv module.some.module.nested1.resource_type.old_name module.some.module.nested1.resource_type.new_name; \
mv module.some.module.nested2.resource_type.old_name module.some.module.nested2.resource_type.new_name; \
mv module.other.module.nested1.resource_type.old_name module.other.module.nested1.resource_type.new_name; \
mv module.other.module.nested2.resource_type.old_name module.other.module.nested2.resource_type.new_name; \
end

  Acquiring lock...
    Move to module.some.module.nested1.resource_type.new_name succeeded
    Move to module.some.module.nested1.resource_type.new_name succeeded
    Move from module.other.module.nested1.resource_type.old_name failed
    Move to module.other.module.nested2.resource_type.new_name succeeded
  Releasing lock...

In large projects with lots of moves, Terraform can end up spending a lot of time releasing and re-acquiring the lock between steps.

References

Similar:

crw commented 2 years ago

Thanks for the report!

dekimsey commented 2 years ago

To add some context, what makes bulk moves particularly of interest is to make these bulk actions atomic. Say I need to rename a set of resources, I want to make sure they either all apply successfully or not at all. Perhaps with a new -missing-ok behavior. Currently it's not possible to do this.

I came across this when I was thinking of the implications of hashicorp/vscode-terraform#699.

apparentlymart commented 1 month ago

Although it's not exactly what was proposed in this issue, I expect that at least some folks with this need would benefit from the later-added moved blocks, which allow encoding the refactoring history of a module as part of the module and thus Terraform can adjust the state automatically as part of a normal plan/apply round.

This new approach is also in some senses "atomic", in that it will first explain all of the renamings it intends to make as part of the plan, and then only actually commit them if you apply the plan. A config-driven move can't fail at apply time if it was successfully planned, so this is more-or-less atomic even though Terraform doesn't strictly guarantee to commit all changes made at apply time in a single state update. (In practice the moved-related updates often are as an implementation detail, because they are immediately ready at the start of the apply phase, but that's technically not a guarantee.)

hashicorp / terraform