Errored backend path while using "workspace_key_prefix" option

clemlesne commented 3 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform: 0.14.4
hashicorp/aws: 3.23.0

Affected Resource(s)

backend

Terraform Configuration Files

Main configuration:

terraform {
  # Fix version to ensure future compatibility if an updates is deployed
  required_version = "~> 0.14"

  backend "s3" { }
}

provider "aws" {
  region  = var.awsResourcesRegion
}

CLI command:

terraform \
  -chdir=./ci/deployment/terraform init \
  -upgrade -no-color -lock=true -lock-timeout=10m \
  -backend-config="key=default" \
  -backend-config="encrypt=true" \
  -backend-config="region=eu-central-1" \
  -backend-config="bucket=my-tfstate" \
  -backend-config="kms_key_id=arn:aws:kms:eu-central-1:my-account-id:alias/my-tfstate" \
  -backend-config="dynamodb_table=my-tfstate" \
  -backend-config="workspace_key_prefix=backends/infra"

Envs:

AWS_EC2_METADATA_DISABLED=true
TF_IN_AUTOMATION=non-null
TF_INPUT=false
TF_DATA_DIR=/tmp/xxx
TF_LOG=TRACE
TF_VAR_awsResourcesRegion=eu-central-1

Debug Output

https://gist.github.com/instagendleg/2fdc89b09c6165d391e0db27e9c25be8

Expected Behavior

I defined my workspace_key_prefix backend prefix equals to backends/infra and my key name equals to default, my AWS S3 file should be of type /backends/infra/[workspace]/default.

Actual Behavior

Instead of, my backend is written at the path /default.

Steps to Reproduce

terraform apply

bflad commented 3 years ago

Hi @instagendleg 👋 Thank you for submitting this and sorry you ran into trouble here. The Terraform S3 Backend code is currently part of the upstream Terraform CLI codebase so any issue discussions and potential bug fixes would need to occur there with those familiar with that functionality. I will go ahead and transfer this issue.

clemlesne commented 3 years ago

Hi,

I founded two workarounds to this bug:

Set my key parameter to what I wished to have with workspace_key_prefix, so backends-infra-[workspace]-default.tfstate. This can be automatized.
Do not use default as your workspace name. Using default remove the behavior or workspace_key_prefix. After multiple tests, I renamed my defaults workspaces to standard. Could it be possible for you to specify this properly in the doc?

aaron-ballard-530 commented 3 years ago

I'm still seeing this issue in terraform 1.0.9.

daithi-walker commented 2 years ago

This issue is preventing me from using the same s3 bucket to store multiple terraform repos or groups of state files. I want to be able to break up my infrastructure into products but store the states for each product in the same bucket.

For GCS, this is possible using the prefix key so I am not sure why it's not possible for AWS.

[prefix](https://www.terraform.io/language/settings/backends/gcs#prefix) - (Optional) GCS prefix inside the bucket. Named states for workspaces are stored in an object called <prefix>/<name>.tfstate.

Ref: https://www.terraform.io/language/settings/backends/gcs

hashicorp / terraform