Linter should skip tfvars files

[morgante]

We don't really care about linting tfvars with terraform validate and it can cause issues with symlinks (see https://github.com/terraform-google-modules/terraform-google-network/pull/67).

Let's update the linting task in the DevEx image to skip linting on tfvars files.

[aaron-lane]

@omazin please take this on.

[omazin]

Investigated, the thing is terraform fmt (https://github.com/GoogleCloudPlatform/cloud-foundation-toolkit/blob/master/infra/build/developer-tools/build/scripts/task_helper_functions.sh#L120) checks both configuration files (.tf) and variables files (.tfvars). If they provide a way to exclude the .tfvars files then the issue will be solved. Or if they implement excluding by file header feature, see https://github.com/hashicorp/terraform/issues/22582, so the linter could ignore such files.

[omazin]

@aaron-lane The PR's https://github.com/terraform-google-modules/terraform-google-network/pull/67 linter build fails with the following log:

Checking for documentation generation
diff: /workspace/test/fixtures/all_examples/terraform.tfvars: No such file or directory
diff: /tmp/tmp.E1dCVAPvAa/generate_docs/workspace/test/fixtures/all_examples/terraform.tfvars: No such file or directory
diff: /workspace/test/fixtures/delete_default_gateway_routes/terraform.tfvars: No such file or directory
diff: /tmp/tmp.E1dCVAPvAa/generate_docs/workspace/test/fixtures/delete_default_gateway_routes/terraform.tfvars: No such file or directory
diff: /workspace/test/fixtures/multi_vpc/terraform.tfvars: No such file or directory
diff: /tmp/tmp.E1dCVAPvAa/generate_docs/workspace/test/fixtures/multi_vpc/terraform.tfvars: No such file or directory
diff: /workspace/test/fixtures/secondary_ranges/terraform.tfvars: No such file or directory
diff: /tmp/tmp.E1dCVAPvAa/generate_docs/workspace/test/fixtures/secondary_ranges/terraform.tfvars: No such file or directory
diff: /workspace/test/fixtures/simple_project/terraform.tfvars: No such file or directory
diff: /tmp/tmp.E1dCVAPvAa/generate_docs/workspace/test/fixtures/simple_project/terraform.tfvars: No such file or directory
diff: /workspace/test/fixtures/simple_project_with_regional_network/terraform.tfvars: No such file or directory
diff: /tmp/tmp.E1dCVAPvAa/generate_docs/workspace/test/fixtures/simple_project_with_regional_network/terraform.tfvars: No such file or directory
diff: /workspace/test/fixtures/submodule_firewall/terraform.tfvars: No such file or directory
diff: /tmp/tmp.E1dCVAPvAa/generate_docs/workspace/test/fixtures/submodule_firewall/terraform.tfvars: No such file or directory
Error: Documentation generation has not been run, please run the
'make docker_generate_docs' command and commit the above changes.
Checking for trailing whitespace
Checking for missing newline at end of file
Running shellcheck
Checking file headers
Running flake8
Running terraform fmt
terraform fmt -diff -check=true -write=false .
terraform fmt -diff -check=true -write=false ./codelabs/simple
terraform fmt -diff -check=true -write=false ./examples/delete_default_gateway_routes
terraform fmt -diff -check=true -write=false ./examples/multi_vpc
terraform fmt -diff -check=true -write=false ./examples/secondary_ranges
terraform fmt -diff -check=true -write=false ./examples/simple_project
terraform fmt -diff -check=true -write=false ./examples/simple_project_with_regional_network
terraform fmt -diff -check=true -write=false ./examples/submodule_firewall
terraform fmt -diff -check=true -write=false ./examples/submodule_svpc_access
terraform fmt -diff -check=true -write=false ./modules/fabric-net-firewall
terraform fmt -diff -check=true -write=false ./modules/fabric-net-svpc-access
terraform fmt -diff -check=true -write=false ./test/fixtures/all_examples

Error: Failed to read file test/fixtures/all_examples/terraform.tfvars

terraform fmt -diff -check=true -write=false ./test/fixtures/delete_default_gateway_routes

Error: Failed to read file test/fixtures/delete_default_gateway_routes/terraform.tfvars

terraform fmt -diff -check=true -write=false ./test/fixtures/multi_vpc

Error: Failed to read file test/fixtures/multi_vpc/terraform.tfvars

terraform fmt -diff -check=true -write=false ./test/fixtures/secondary_ranges

Error: Failed to read file test/fixtures/secondary_ranges/terraform.tfvars

terraform fmt -diff -check=true -write=false ./test/fixtures/shared
terraform fmt -diff -check=true -write=false ./test/fixtures/simple_project

Error: Failed to read file test/fixtures/simple_project/terraform.tfvars

terraform fmt -diff -check=true -write=false ./test/fixtures/simple_project_with_regional_network

Error: Failed to read file test/fixtures/simple_project_with_regional_network/terraform.tfvars

terraform fmt -diff -check=true -write=false ./test/fixtures/submodule_firewall

Error: Failed to read file test/fixtures/submodule_firewall/terraform.tfvars

terraform fmt -diff -check=true -write=false ./test/setup
Error: terraform fmt failed with exit code 123
Check the output for diffs and correct using terraform fmt <dir>
Error: The following tests have failed: check_documentation check_terraform
ERROR
ERROR: build step 0 "gcr.io/cloud-foundation-cicd/cft/developer-tools:0.1.0" failed: exit status 124

It fails because it can't find the file test/fixtures/shared/terraform.tfvars which is referenced by a symlink in some terraform.tfvars files. terraform fmt and rsync commands fail in check_terraform and check_documentation tests respectively. Made a pull request https://github.com/GoogleCloudPlatform/cloud-foundation-toolkit/pull/401 that excludes tfvars files from the linting tasks. I think the fix should be merged, developer tools image's tag updated and changed in the network module respectively https://github.com/terraform-google-modules/terraform-google-network/blob/master/build/lint.cloudbuild.yaml#L24