Workspace-specific Automatic Variable Values

[RichardN]

In order to ease configuration of multiple environments When I have selected an environment using terraform env select development It would be good Terraform could automatically do the equivalent of --var-file=development.tfvars

This would enable values that change across environments to be easily configured without lots of 'if' settings in the terraform config.

[nbering]

Personally, I solve this problem with a wrapper script around terraform that sources a .env file with a bunch of TF_VAR_whatever environment variables (including my provider access keys). It's not perfect, but it saves the conditionals and I don't have to wait for a new feature.

[mitchellh]

This is a planned improvement, will tag. It may not be exactly this but we do plan on environment-specific vars.

[RichardN]

Excellent. I don't mind how you do it, just that it is easy to understand and use!

[joshuaspence]

+1 on this. I also have a somewhat related use case. We have a bunch of different stacks that live in a common directory. We have some configuration that is local to a stack, but also some configuration which is shared.

Basically, at the moment we run terraform with -var-file=../terraform.tfvars -var-file=terraform.tfvars (note that we do need to explicitly include terraform.tfvars so that variables defined in terraform.tfvars can override variables from ../terraform.tfvars).

I think that a more general solution could be used to solve both of these problems. Maybe we could allowed the variable files to be defined in Terraform itself? Something like this:

terraform {
  variable_files = [
    "${path.cwd}/terraform.${terraform.environment}.tfvars",
    "${path.cwd}/terraform.tfvars",
    "${path.cwd}/../terraform.tfvars",
  ]
}

[bradenwright]

FYI Terragrunt supports this type of feature currently, see arguments block: https://github.com/gruntwork-io/terragrunt#multiple-extra_arguments-blocks

[matti]

see a possible workaround: https://github.com/hashicorp/terraform/issues/15966#issuecomment-359766835

[peimanja]

+1

[gaui]

This is a crucial feature for multi environment Terraform setup. I don't want to use Makefile or Terragrunt.

[sstarcher]

Highly looking forward to this feature. We are currently not using workspaces due to the lack of this feature.

[lkysow]

Terraform workspace docs have been updated to discourage using workspaces for environments so I assume this is no longer the direction that will be taken?

In particular, organizations commonly want to create a strong separation between multiple deployments of the same infrastructure serving different development stages (e.g. staging vs. production) or different internal teams. In this case, the backend used for each deployment often belongs to that deployment, with different credentials and access controls. Named workspaces are not a suitable isolation mechanism for this scenario.

From https://www.terraform.io/docs/state/workspaces.html#when-to-use-multiple-workspaces

[gaui]

The proposed alternative is really bad. Gonna use workspaces until this is improved and some clear alternative is proposed.

[andrew-j-hagner]

Terraform workspace docs have been updated to discourage using workspaces for environments so I assume this is no longer the direction that will be taken?

In particular, organizations commonly want to create a strong separation between multiple deployments of the same infrastructure serving different development stages (e.g. staging vs. production) or different internal teams. In this case, the backend used for each deployment often belongs to that deployment, with different credentials and access controls. Named workspaces are not a suitable isolation mechanism for this scenario.

From https://www.terraform.io/docs/state/workspaces.html#when-to-use-multiple-workspaces

@lkysow In the case of using S3 for the backend, workspaces are still a suitable option because you can apply permissions/access per file, right? If so it might make sense to only recommend not using workspaces if the backing store doesn't support strong separation itself. The current docs make it sound as if you can't/shouldn't use workspaces if you want strong separation (different access/permission controls) around the different workspace states. But you can still get that with some backends like S3. This makes workspaces sound a lot less useful, as soon as you want strong separation, which seems to be a common practice, you have to abandon the builtin workspace workflow and go back to separate configs per environment which seems like a lot of extra boiler plate.

[lkysow]

You're talking about access/permissions for the statefiles. I thought that the docs were referring to access/permissions for the provider itself.

For example if you have two AWS accounts for staging/prod with different credentials, then using a workspace for staging and a workspace for prod doesn't help you deal with using separate credentials for each workspace. You'd still need to switch creds manually (locally) or use terraform init -backend-config. There's no way to say "use these creds when I'm in workspace staging and use these creds when I'm in workspace production".

[andrew-j-hagner]

ah ok, that makes a little more sense. That scenario seems a little strange but I can see it, and it does seem to lower the value of workspaces significantly for those users.

---

There's no way to say "use these creds when I'm in workspace staging and use these creds when I'm in workspace production".

My thought would be to use a build tool and set the appropriate creds for the workspace being updated before running terraform. I.e. It's the build tool's responsibility to run as/supply the right creds for the workspace being updated. (noted that if you have to use completely separate backends for state per env this still doesn't work)

So a sample workflow could look something like: 1) Developer makes changes to terraform code in a private terraform workspace, applys, tests, code reviews, and merges into development branch. 2) Commit triggers development build, which after other steps, provides creds for development environment backend (the dev state file specifically), runs terraform for development workspace with dev.tfvars, and then does any final steps. 3) At some point development is merged into master, same build process happens but this time providing creds for production and prod.tfvars.

In this case: 1) Same terraform code is used for all envs 2) Permission to apply in each environment can be controlled (assuming backend supports per file acl) 3) Build server provides correct creds depending on environment being deployed 4) Environment differences managed by tfvars, not duplicated terraform code (and thus this github issue would be a nice to have implemented, along with the rest of https://github.com/hashicorp/terraform/issues/16342 )

[apparentlymart]

Along with what's already been said, there's also a way to use workspaces across accounts with the S3 backend described in the docs. Although as we currently stand I would still suggest that a separate config per environment with shared modules is the most flexible solution (echoing what is in the "When to use Workspaces" section), using assume_role as an indirection to separate the backend auth from the provider auth while still only distributing one set of credentials is a pattern used by a lot of folks in the wild, and while it does take some work to get it set up it seems to be reasonably easy to use day-to-day once you've done that initial work.

With that said, we're aware that the current workspaces feature is not quite hitting the spot for what is commonly needed, and the "When to use Workspaces" section is intended to acknowledge that and suggest other approaches that work better with Terraform as it exists today.

We are intending to revisit the design this part of Terraform in a future release to improve the usability for the common case of working across multiple entirely-separate environments, but we (the Terraform team at HashiCorp) are currently focused on improvements in the area of configuration and modules, which we want to complete before getting into the details of adjusting the backends/workspaces design just because we want to let those configuration language changes (starting in the forthcoming v0.12.0 release, and continuing in some subsequent minor ones) settle first.

[lkysow]

Ahh, the S3 + workspaces solution looks pretty good actually. I didn't know about that. Thanks!

[EnzoDotjs]

Just a suggestion to solve this is why not automatically load file names .workspace.tfvars similiar to how all *.auto.tfvars are automatically loaded.

I saw another issue https://github.com/hashicorp/terraform/issues/29450 which the gentleman just reverted to using something like this always:

terraform plan -var-file=$(terraform workspace show).tfvars

and that is better but if terraform would automatically load the proper tfvars file it would avoid that pesky typing.

I would think you would only need to grab the code for *.auto.tfvars and refactor it looking for that specific file .workspace.tfvars.

[ashleyjkell]

Having to hack around this or use Terragrunt in 2023 is frustrating at best, is there any timeline on this getting implemented?