terraform-google-modules / terraform-example-foundation

Shows how the CFT modules can be composed to build a secure cloud foundation
https://cloud.google.com/architecture/security-foundations
Apache License 2.0
1.23k stars 720 forks source link

3-networks - The root module does not declare a variable named "target_name_server_addresses" #169

Closed caleonardo closed 4 years ago

caleonardo commented 4 years ago

Please note I am opening two bugs, both errors appear in the log below

ERROR:

The root module does not declare a variable named
"target_name_server_addresses" but a value was found in file
"terraform.tfvars". To use this value, add a "variable" block to the
configuration.

CAUSE: The variables in 3-networks/terraform.example.tfvars are the same for all envs and the instructions only ask the user to rename the file.

All envs/ in 3-networks share the same terraform.example.tfvars - However, the envs/shared uses different variables that envs/dev, envs/prod and envs/nonpord

POSSIBLE SOLUTION The instructions should ask the user to make two copies of the file: one for 3-networks/envs/shared and one for 3-networks/envs/prod|nonprod|dev

Note that you can also see the error described in https://github.com/terraform-google-modules/terraform-example-foundation/issues/170 - they seem to be two different issues

====================+====================+====================+====================+======

FailedConsole Output Branch indexing

git rev-parse --is-inside-work-tree # timeout=10 Setting origin to https://github.com/caleonardo/cft-01-test-3-networks git config remote.origin.url https://github.com/caleonardo/cft-01-test-3-networks # timeout=10 Fetching origin... Fetching upstream changes from origin git --version # timeout=10 git config --get remote.origin.url # timeout=10 git fetch --tags --progress -- origin +refs/heads/:refs/remotes/origin/ # timeout=10 Seen branch in repository origin/master Seen branch in repository origin/plan Seen 2 remote branches Obtained Jenkinsfile from d7ed833b4f406509e2da292357f4aa434d80450e Running in Durability level: MAX_SURVIVABILITY [Pipeline] Start of Pipeline [Pipeline] node Running on jenkins-ssh-agent-Agent1-GCE-Bootstrap-1 in /home/jenkins/jenkins_agent_dir/workspace/line-cft-01-test-3-networks_plan [Pipeline] { [Pipeline] stage [Pipeline] { (Declarative: Checkout SCM) [Pipeline] checkout No credentials specified Cloning the remote Git repository Cloning with configured refspecs honoured and without tags Cloning repository https://github.com/caleonardo/cft-01-test-3-networks git init /home/jenkins/jenkins_agent_dir/workspace/line-cft-01-test-3-networks_plan # timeout=10 Fetching upstream changes from https://github.com/caleonardo/cft-01-test-3-networks git --version # timeout=10 git fetch --no-tags --progress -- https://github.com/caleonardo/cft-01-test-3-networks +refs/heads/:refs/remotes/origin/ # timeout=10 Fetching without tags git config remote.origin.url https://github.com/caleonardo/cft-01-test-3-networks # timeout=10 git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 git config remote.origin.url https://github.com/caleonardo/cft-01-test-3-networks # timeout=10 Fetching upstream changes from https://github.com/caleonardo/cft-01-test-3-networks git fetch --no-tags --progress -- https://github.com/caleonardo/cft-01-test-3-networks +refs/heads/:refs/remotes/origin/ # timeout=10 Checking out Revision d7ed833b4f406509e2da292357f4aa434d80450e (plan) Commit message: "Initial commit for the 3-networks step" First time build. Skipping changelog. git config core.sparsecheckout # timeout=10 git checkout -f d7ed833b4f406509e2da292357f4aa434d80450e # timeout=10 git --version # timeout=10 [Pipeline] } [Pipeline] // stage [Pipeline] withEnv [Pipeline] { [Pipeline] withEnv [Pipeline] { [Pipeline] stage [Pipeline] { (setup) [Pipeline] sh

  • echo Setting up gcloud for impersonation Setting up gcloud for impersonation
  • gcloud config set auth/impersonate_service_account org-terraform@cft-seed-1437.iam.gserviceaccount.com Updated property [auth/impersonate_service_account].
  • echo Adding bucket information to backends Adding bucket information to backends
  • find -name backend.tf
  • sed -i s/UPDATE_ME/cft-tfstate-c32b/ ./envs/prod/backend.tf
  • sed -i s/UPDATE_ME/cft-tfstate-c32b/ ./envs/dev/backend.tf
  • sed -i s/UPDATE_ME/cft-tfstate-c32b/ ./envs/nonprod/backend.tf
  • sed -i s/UPDATE_ME/cft-tfstate-c32b/ ./envs/shared/backend.tf [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (TF plan validate all) [Pipeline] sh
  • ./tf-wrapper.sh plan_validate_all plan * TERRAFORM INIT ***** At environment: envs/prod

    Initializing modules...

  • private_shared_vpc in ../../modules/standard_shared_vpc Downloading terraform-google-modules/network/google 2.4.0 for private_shared_vpc.main...
  • private_shared_vpc.main in .terraform/modules/private_shared_vpc.main/terraform-google-network-2.4.0
  • private_shared_vpc.main.routes in .terraform/modules/private_shared_vpc.main/terraform-google-network-2.4.0/modules/routes
  • private_shared_vpc.main.subnets in .terraform/modules/private_shared_vpc.main/terraform-google-network-2.4.0/modules/subnets
  • private_shared_vpc.main.vpc in .terraform/modules/private_shared_vpc.main/terraform-google-network-2.4.0/modules/vpc Downloading terraform-google-modules/cloud-dns/google 3.0.2 for private_shared_vpc.peering_zone...
  • private_shared_vpc.peering_zone in .terraform/modules/private_shared_vpc.peering_zone/terraform-google-cloud-dns-3.0.2 Downloading terraform-google-modules/cloud-dns/google 3.0.2 for private_shared_vpc.private_gcr...
  • private_shared_vpc.private_gcr in .terraform/modules/private_shared_vpc.private_gcr/terraform-google-cloud-dns-3.0.2 Downloading terraform-google-modules/cloud-dns/google 3.0.2 for private_shared_vpc.private_googleapis...
  • private_shared_vpc.private_googleapis in .terraform/modules/private_shared_vpc.private_googleapis/terraform-google-cloud-dns-3.0.2 Downloading terraform-google-modules/cloud-router/google 0.2.0 for private_shared_vpc.region1_router1...
  • private_shared_vpc.region1_router1 in .terraform/modules/private_shared_vpc.region1_router1/terraform-google-cloud-router-0.2.0 Downloading terraform-google-modules/cloud-router/google 0.2.0 for private_shared_vpc.region1_router2...
  • private_shared_vpc.region1_router2 in .terraform/modules/private_shared_vpc.region1_router2/terraform-google-cloud-router-0.2.0 Downloading terraform-google-modules/cloud-router/google 0.2.0 for private_shared_vpc.region2_router1...
  • private_shared_vpc.region2_router1 in .terraform/modules/private_shared_vpc.region2_router1/terraform-google-cloud-router-0.2.0 Downloading terraform-google-modules/cloud-router/google 0.2.0 for private_shared_vpc.region2_router2...
  • private_shared_vpc.region2_router2 in .terraform/modules/private_shared_vpc.region2_router2/terraform-google-cloud-router-0.2.0
  • restricted_shared_vpc in ../../modules/restricted_shared_vpc Downloading terraform-google-modules/vpc-service-controls/google 2.0.0 for restricted_shared_vpc.access_level_members...
  • restricted_shared_vpc.access_level_members in .terraform/modules/restricted_shared_vpc.access_level_members/terraform-google-vpc-service-controls-2.0.0/modules/access_level Downloading terraform-google-modules/network/google 2.4.0 for restricted_shared_vpc.main...
  • restricted_shared_vpc.main in .terraform/modules/restricted_shared_vpc.main/terraform-google-network-2.4.0
  • restricted_shared_vpc.main.routes in .terraform/modules/restricted_shared_vpc.main/terraform-google-network-2.4.0/modules/routes
  • restricted_shared_vpc.main.subnets in .terraform/modules/restricted_shared_vpc.main/terraform-google-network-2.4.0/modules/subnets
  • restricted_shared_vpc.main.vpc in .terraform/modules/restricted_shared_vpc.main/terraform-google-network-2.4.0/modules/vpc Downloading terraform-google-modules/cloud-dns/google 3.0.2 for restricted_shared_vpc.peering_zone...
  • restricted_shared_vpc.peering_zone in .terraform/modules/restricted_shared_vpc.peering_zone/terraform-google-cloud-dns-3.0.2 Downloading terraform-google-modules/cloud-router/google 0.2.0 for restricted_shared_vpc.region1_router1...
  • restricted_shared_vpc.region1_router1 in .terraform/modules/restricted_shared_vpc.region1_router1/terraform-google-cloud-router-0.2.0 Downloading terraform-google-modules/cloud-router/google 0.2.0 for restricted_shared_vpc.region1_router2...
  • restricted_shared_vpc.region1_router2 in .terraform/modules/restricted_shared_vpc.region1_router2/terraform-google-cloud-router-0.2.0 Downloading terraform-google-modules/cloud-router/google 0.2.0 for restricted_shared_vpc.region2_router1...
  • restricted_shared_vpc.region2_router1 in .terraform/modules/restricted_shared_vpc.region2_router1/terraform-google-cloud-router-0.2.0 Downloading terraform-google-modules/cloud-router/google 0.2.0 for restricted_shared_vpc.region2_router2...
  • restricted_shared_vpc.region2_router2 in .terraform/modules/restricted_shared_vpc.region2_router2/terraform-google-cloud-router-0.2.0 Downloading terraform-google-modules/cloud-dns/google 3.0.2 for restricted_shared_vpc.restricted_gcr...
  • restricted_shared_vpc.restricted_gcr in .terraform/modules/restricted_shared_vpc.restricted_gcr/terraform-google-cloud-dns-3.0.2 Downloading terraform-google-modules/cloud-dns/google 3.0.2 for restricted_shared_vpc.restricted_googleapis...
  • restricted_shared_vpc.restricted_googleapis in .terraform/modules/restricted_shared_vpc.restricted_googleapis/terraform-google-cloud-dns-3.0.2

Initializing the backend...  Successfully configured the backend "gcs"! Terraform will automatically use this backend unless the backend configuration changes.

Initializing provider plugins...

The following providers do not have any version constraints in configuration, so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking changes, it is recommended to add version = "..." constraints to the corresponding provider blocks in configuration, with the constraint strings suggested below.

Terraform has been successfully initialized!  You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work.

If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary. * TERRAFORM PLAN ***** At environment: envs/prod


Refreshing Terraform state in-memory prior to plan... The refreshed state will be used to calculate this plan, but will not be persisted to local or remote state storage.  data.google_service_account_access_token.default: Refreshing state... module.private_shared_vpc.data.google_netblock_ip_ranges.iap_forwarders: Refreshing state... module.restricted_shared_vpc.data.google_netblock_ip_ranges.health_checkers: Refreshing state... module.restricted_shared_vpc.data.google_netblock_ip_ranges.legacy_health_checkers: Refreshing state... data.google_active_folder.env: Refreshing state... module.private_shared_vpc.data.google_netblock_ip_ranges.health_checkers: Refreshing state... module.restricted_shared_vpc.data.google_netblock_ip_ranges.iap_forwarders: Refreshing state... module.restricted_shared_vpc.data.google_active_folder.common: Refreshing state... module.private_shared_vpc.data.google_netblock_ip_ranges.legacy_health_checkers: Refreshing state... module.private_shared_vpc.data.google_active_folder.common: Refreshing state... module.private_shared_vpc.data.google_projects.dns_hub: Refreshing state... module.restricted_shared_vpc.data.google_projects.dns_hub: Refreshing state... data.google_projects.restricted_host_project: Refreshing state... data.google_projects.private_host_project: Refreshing state... data.google_project.restricted_host_project: Refreshing state... module.restricted_shared_vpc.data.google_compute_network.vpc_dns_hub: Refreshing state... module.private_shared_vpc.data.google_compute_network.vpc_dns_hub: Refreshing state...


 Warning: Value for undeclared variable

The root module does not declare a variable named "target_name_server_addresses" but a value was found in file "terraform.tfvars". To use this value, add a "variable" block to the configuration.

Using a variables file to set an undeclared variable is deprecated and will become an error in a future release. If you wish to provide certain "global" settings to all configurations in your organization, use TFVAR... environment variables to set these instead.   Error: "peering_config.0.target_network.0.network_url": required field is not set

 on .terraform/modules/private_shared_vpc.peering_zone/terraform-google-cloud-dns-3.0.2/main.tf line 21, in resource "google_dns_managed_zone" "peering": 21: resource "google_dns_managed_zone" "peering" {    Error: "peering_config.0.target_network.0.network_url": required field is not set

 on .terraform/modules/restricted_shared_vpc.peering_zone/terraform-google-cloud-dns-3.0.2/main.tf line 21, in resource "google_dns_managed_zone" "peering": 21: resource "google_dns_managed_zone" "peering" {   [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (TF init) Stage "TF init" skipped due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (TF plan) Stage "TF plan" skipped due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (TF validate) Stage "TF validate" skipped due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (TF apply) Stage "TF apply" skipped due to earlier failure(s) [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline ERROR: script returned exit code 21 Finished: FAILURE

caleonardo commented 4 years ago

Another possible solution is to use multiple variable files, but then we need to change the tf-wrapper to be aware of that, as in:

For envs/shared: $ terraform plan –var-file terraform.common.tfvars –var-file terraform.shared.tfvars

For envs/dev|prod|nonprod: $ terraform plan –var-file terraform.common.tfvars or point the terraform.tfvars@ link to terraform.common.tfvars

daniel-cit commented 4 years ago

in Terraform v0.12.24 this only give a warning, not an error. Which version of terraform are you using ?

daniel-cit commented 4 years ago

Something like was done in 4-projects could be done here too:

to have in 3-networks:

and on 3-networks/envs/[dev|nonprod|prod] have

commons.auto.tfvar -> ../../commons.auto.tfvar

and on 3-networks/envs/shared have

commons.auto.tfvar -> ../../commons.auto.tfvar shared.auto.tfvar -> ../../shared.auto.tfvar

amandakarina commented 4 years ago

PR merged