[FEATURE]: More intuitive terraform plugin CLI

[stefreak]

Feature Request

A more intuitive terraform plugin CLI

Background / Motivation

• Garden Terraform users need to remember yet another Command line interface that needlessly differs from the terraform CLI itself.
• Even users who use autoApply need to run terraform commands from time to time, for example to inspect or modify the state. Currently they need to remember or look up the terraform version, and run garden tools terraform.terraform-<terraform-version> -- some terraform command.
• If the terraform stack relies on variables in the Garden project, and the user wants to run custom commands, the user needs to remember to run garden plugins terraform plan-root/plan-module before running garden tools terraform.terraform-<terraform-version> -- some terraform command
• The plugin command structure is very hard to remember. For example, applying a particular module looks like this right now: garden --env my-service-prod-eu plugins terraform plan-module -- terraform-module

What should the user be able to do?

I would suggest that we change our CLI, for example in Garden 0.13, to be, where possible, exactly the same as terraform's CLI.

Terraform module-level config examples

How to select a module:

-garden plugins terraform plan-module -- terraform-module
+garden plugins terraform --module=terraform-module plan

Or, if there is only one terraform module, just:

-garden plugins terraform plan-module -- terraform-module
+garden plugins terraform plan

Provider-level terraform config examples

-garden plugins terraform plan-root
+garden plugins terraform plan

In case of ambiguity, for example if there is also a module in addition to the provider-level terraform stack, garden could list the possible options to the user:

Error: There are multiple terraform configurations available in this Garden project:
- `./common-infra` is declared at the provider level
- `./dev-module` is a terraform module called `dev-infra`

You have these options:

- Run `garden plugins terraform --root apply` if you want to use the provider-level terraform stack `./common-infra`
- Run `garden plugins terraform --module=dev-infra apply` to use the stack in the directory `./dev-module`

Help

It would be nice to actually forward the --help flag to the terraform command and list all the available options for the current version.

-% garden plugins terraform plan-root --help
-DESCRIPTION
-
-Execute a command defined by a plugin in your project.
-Run without arguments to get a list of all plugin commands available.
-Run with just the plugin name to get a list of commands provided by that plugin.
-
-Examples:
-
-    # Run the `cleanup-cluster-registry` command from the `kubernetes` plugin.
-    garden plugins kubernetes cleanup-cluster-registry
-
-    # List all available commands.
-    garden plugins
-
-    # List all the commands from the `kubernetes` plugin.
-    garden plugins kubernetes
-
-USAGE
-  garden plugins [plugin] [command] [options]
-
-ARGUMENTS
-  [command]  The name of the command to run.
-             [string]
-  [plugin]   The name of the plugin, whose command you wish to run.
-             [string]
+% garden plugins terraform plan --help
+Usage: terraform plan [options] [DIR]
+
+  Generates an execution plan for Terraform.
+
+  This execution plan can be reviewed prior to running apply to get a
+  sense for what Terraform will do. Optionally, the plan can be saved to
+  a Terraform plan file, and apply can take this plan file to execute
+  this plan exactly.
+
+Options:
+
+  -compact-warnings   If Terraform produces any warnings that are not
+                      accompanied by errors, show them in a more compact form
+                      that includes only the summary messages.
+
+  -destroy            If set, a plan will be generated to destroy all resources
+                      managed by the given configuration and state.
+
+  -detailed-exitcode  Return detailed exit codes when the command exits. This
+                      will change the meaning of exit codes to:
+                      0 - Succeeded, diff is empty (no changes)
+                      1 - Errored
+                      2 - Succeeded, there is a diff
+
+  -input=true         Ask for input for variables if not directly set.
+
+  -lock=true          Lock the state file when locking is supported.
+
+  -lock-timeout=0s    Duration to retry a state lock.
+
+  -no-color           If specified, output won't contain any color.
+
+  -out=path           Write a plan file to the given path. This can be used as
+                      input to the "apply" command.
+
+  -parallelism=n      Limit the number of concurrent operations. Defaults to 10.
+
+  -refresh=true       Update state prior to checking for differences.
+
+  -state=statefile    Path to a Terraform state file to use to look
+                      up Terraform-managed resources. By default it will
+                      use the state "terraform.tfstate" if it exists.
+
+  -target=resource    Resource to target. Operation will be limited to this
+                      resource and its dependencies. This flag can be used
+                      multiple times.
+
+  -var 'foo=bar'      Set a variable in the Terraform configuration. This
+                      flag can be set multiple times.
+
+  -var-file=foo       Set variables in the Terraform configuration from
+                      a file. If "terraform.tfvars" or any ".auto.tfvars"
+                      files are present, they will be automatically loaded.

Shell

In case the user wants to perform a series of task using the terraform executable directly, we could also offer a terraform shell.

Example session:

% garden plugins terraform shell
[...]
Garden terraform shell: Changed directory to /myproject/terraform-module/ and added /Users/steffen/.garden/tools/terraform-1-3-7/ff4233ee4ef39b40/ to the PATH environment variable.
Launching /bin/zsh
% which terraform
/Users/steffen/.garden/tools/terraform-1-3-7/ff4233ee4ef39b40/terraform
% ls
garden.yml main.tf

Why do they want to do this? What problem does it solve?

Terraform users need to interact with terraform directly in some cases. Garden should make this easy, and not harder than it should be.

Experienced terraform users should not need to learn the kinks of yet another CLI. It should feel as familiar as possible to them.

Suggested Implementation(s)

Did not look into the code, but changing the CLI should be doable. There might be quirks in the current way we parse CLI arguments.

How important is this feature for you/your team?

🌵 Not having this feature makes using Garden painful

[johnlane]

This would be very useful. I think the restructured commands are much more intuitive and will commit to memory much easier. And the idea of a shell where you can run terraform directly with the relevant environment preset is great. It will make it much easier to do simple things such as terraform output, working with the Terraform state e.g. terraform state list , terraform state mv ... , resource-targeted applies and so-on.

[vvagaytsev]

@edvald this seems to be a breaking change. Should we address it in 0.13?

[vvagaytsev]

I added the 0.13 label. Let's decide on this before the official 0.13 release.