Terraform not loading auto.tfvars when executed with a directory

[mkielar]

Terraform Version

→ terraform version
Terraform v0.12.26

Terraform Configuration Files

See terraform_test_case.zip

Expected Behavior

Terraform should load the dev.auto.tfvars file from specified path when ran with directory argument.

Actual Behavior

When I run terraform apply <dir>, terraform does not automatically load auto.tfvars in that dir and instead asks me to provide values for variables:

terraform_test_case>terraform apply terraform/environments/dev
terraform apply terraform/environments/dev
var.foo
  Enter a value:

Steps to Reproduce

> unzip terraform_test_case.zip
> cd terraform_test_case
terraform_test_case> terraform apply terraform/environments/dev

[apparentlymart]

Hi @mkielar,

In cases where you run Terraform against a configuration somewhere other than the current working directory, Terraform still looks in the current working directory for non-configuration files like the .auto.tfvars files and the .terraform directory. The command line option to specify the configuration directory changes only where Terraform looks for the root module.

For that reason, it seems like Terraform is working as intended here: I expect Terraform would still read the .auto.tfvars files if you had them in your current working directory.

We're not intending to change this behavior because the capability to select a different configuration directory than the current working directory is already largely a legacy capability, which has a number of limitations. The main supported way to run Terraform is to switch current working directory to the root module directory first and then run terraform init and terraform apply from there. If your current working directory and your current module directory are distinct then you will encounter various situations like this where certain Terraform features are defined to look in the current working directory, rather than in the root module directory.

[mkielar]

Legacy. Hm. That's bad news, but let me explain the use case anyway.

We're running multiple validations of Terraform code in our CICD Pipeline (terraform validate, tflint, tfsec). Some of those validators (e.g. tflint with --deep option) require terraform workspace to be initialized in order to perform checks against real infrastructure.

Then they report errors, relative to the directory they are called from. An example with terraform validate would be:

terraform_test_case> terraform validate terraform/environments/dev/

Error: Argument or block definition required

  on terraform/environments/dev/variables.tf line 2:
   2: asdf

An argument or block definition is required here. To set an argument, use the
equals sign "=" to introduce the argument value.

versus

terraform_test_case/terraform/environments/dev>terraform validate

Error: Argument or block definition required

  on variables.tf line 2:
   2: asdf

An argument or block definition is required here. To set an argument, use the
equals sign "=" to introduce the argument value.

Note in the second output, the path to variables.tf file is completely confusing and ambiguous, considering I have more environments in my directory structure.

tflint and tfsec behave the same way when ran inside / outside of the terraform root directory. This makes it impossible (or very hard, and requiring some awkward hacks) to bind those error outputs with tools like reviewdog, which could automate code review comments.

For tflint deep checks this additionally means, I'd have to run terraform init twice. Once from outside, for tflint, then again, from inside for terraform apply so that the latter would load the auto.tfvars files properly.

This is just awkward, and considering your "legacy" comment, I'm wondering if the tflint scenario will at all be possible / supported in the near future. Which makes me wonder what would be a proper configuration for the scenario I described...

[pradeep151287]

Is there a way where we can mention to read terraform.tfvars or *.auto.tfvars file which is there in the target TF code folder, instead of looking at the current folder from where I'm running the terraform command

I have terraform.tfvars on target folder "terraform_code/AWS_ASG"

as of now terraform reads terraform.tfvars or *.auto.tfvars form current directory from where we are running terraform

we are using python script as a wrapper to calling multiple Environment resource, we fail to use tfvars because of current limitations.

[tarfeef102]

I agree with the above, we also have a use case where being able to say "look in the directory passed for the var files instead of the working directory" would be very valuable. It is also what I, at least, intuitively thought the behaviour would be without digging through the docs and interpreting them very literally.

[mkielar]

https://github.com/hashicorp/terraform/pull/26087 looks very promising for this issue, assuming that it the -chdir option will also affect how the auto.tfvars files are loaded.

If someone can confirm that -chdir will load the auto.tfvars from the specified directory, not from the current one, this issue could be closed and we'll wait for 0.14 release to do what we need.