terraform init fails with conflicting module version requirements for hashicorp/google and hashicorp/google-beta

danscarf commented 6 months ago

TL;DR

terraform init fails with conflicting module version requirements
Affected providers are hashicorp/google and hashicorp/google-beta
Trying to get the TF Cloud path working (not CloudBuild or Jenkins)

Expected behavior

Terraform init command should complete without error

Observed behavior

 ~/src/github/terraform-google-modules/gcp-bootstrap/envs/shared  plan ?1  terraform init
Initializing the backend...
Initializing modules...

- Installed hashicorp/kubernetes v2.26.0 (signed by HashiCorp)

 Error: Failed to query available provider packages

 Could not retrieve the list of available versions for provider hashicorp/google-beta: no available releases match the given constraints >= 3.1.0, >= 3.43.0, >=
 3.50.0, >= 3.67.0, >= 3.77.0, ~> 4.5, >= 4.11.0, >= 4.28.0, >= 4.40.0, >= 4.64.0, < 5.0.0, >= 5.6.0, >= 5.9.0, < 6.0.0

 Error: Failed to query available provider packages

 Could not retrieve the list of available versions for provider hashicorp/google: no available releases match the given constraints >= 3.33.0, >= 3.43.0, >= 3.50.0,
│ >= 3.53.0, >= 3.67.0, >= 3.77.0, >= 3.83.0, >= 4.3.0, ~> 4.5, >= 4.25.0, >= 4.28.0, != 4.31.0, >= 4.40.0, >= 4.64.0, < 5.0.0, >= 5.6.0, >= 5.9.0, < 6.0.0

Terraform Configuration

I only modified the terraform.tfvars file and can't include those sensitive values.

 ~/src/github/terraform-google-modules  ls -l
total 0
drwxr-xr-x   5 dan  staff  160 Mar  2 14:26 gcp-bootstrap
drwxr-xr-x   5 dan  staff  160 Mar  2 11:47 gcp-environments
drwxr-xr-x   5 dan  staff  160 Mar  2 11:47 gcp-networks
drwxr-xr-x   5 dan  staff  160 Mar  2 11:47 gcp-org
drwxr-xr-x   5 dan  staff  160 Mar  2 11:47 gcp-projects
drwxr-xr-x  26 dan  staff  832 Mar  2 08:47 terraform-example-foundation
 ~/src/github/terraform-google-modules 

 ~/src/github/terraform-google-modules/gcp-bootstrap  plan ?1  tree                                                                                             ✔
.
└── envs
    └── shared
        ├── Dockerfile
        ├── README-GitHub.md
        ├── README-GitLab.md
        ├── README-Jenkins.md
        ├── README-Terraform-Cloud.md
        ├── README.md
        ├── backend.tf.cloud.example
        ├── backend.tf.example
        ├── cb.tf.example
        ├── files
        │   └── private_key_example.png
        ├── github.tf.example
        ├── gitlab.tf.example
        ├── groups.tf
        ├── jenkins.tf.example
        ├── main.tf
        ├── modules
        │   ├── cb-private-pool
        │   │   ├── README.md
        │   │   ├── main.tf
        │   │   ├── network.tf
        │   │   ├── outputs.tf
        │   │   ├── variables.tf
        │   │   ├── versions.tf
        │   │   └── vpn_ha.tf
        │   ├── gitlab-oidc
        │   │   ├── main.tf
        │   │   ├── outputs.tf
        │   │   ├── variables.tf
        │   │   └── versions.tf
        │   ├── jenkins-agent
        │   │   ├── README.md
        │   │   ├── files
        │   │   │   └── jenkins_gce_startup_script.sh
        │   │   ├── main.tf
        │   │   ├── outputs.tf
        │   │   ├── variables.tf
        │   │   ├── versions.tf
        │   │   └── vpn_ha.tf
        │   ├── parent-iam-member
        │   │   ├── main.tf
        │   │   ├── variables.tf
        │   │   └── versions.tf
        │   ├── parent-iam-remove-role
        │   │   ├── main.tf
        │   │   ├── variables.tf
        │   │   └── versions.tf
        │   └── tfc-agent-gke
        │       ├── README.md
        │       ├── main.tf
        │       ├── outputs.tf
        │       ├── variables.tf
        │       └── versions.tf
        ├── onprem.md
        ├── outputs.tf
        ├── provider.tf
        ├── sa.tf
        ├── scripts
        │   ├── git_create_branches_helper.sh
        │   └── push-to-repo.sh
        ├── terraform.tfvars
        ├── terraform_cloud.tf
        ├── variables.tf
        └── versions.tf

13 directories, 54 files
 ~/src/github/terraform-google-modules/gcp-bootstrap  plan ?1 

Terraform Version

Terraform v1.7.4
on darwin_arm64

Additional information

output of terraform providers command:

 ~/s/github/terraform-google-modules/gcp-bootstrap/envs/shared  plan ?1  terraform providers                                                                  1 ✘

Providers required by configuration:
.
├── provider[registry.terraform.io/hashicorp/google] >= 3.50.0, != 4.31.0
├── provider[registry.terraform.io/hashicorp/tfe] 0.48.0
├── provider[registry.terraform.io/hashicorp/random]
├── provider[registry.terraform.io/hashicorp/google-beta]
├── module.required_group
│   ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.67.0, < 6.0.0
│   └── provider[registry.terraform.io/hashicorp/google] >= 3.67.0, < 6.0.0
├── module.seed_bootstrap
│   ├── provider[registry.terraform.io/hashicorp/google] >= 3.50.0, < 6.0.0
│   ├── provider[registry.terraform.io/hashicorp/random] >= 3.1.0
│   ├── module.enable_cross_project_service_account_usage
│       ├── provider[registry.terraform.io/hashicorp/google] >= 3.53.0, < 6.0.0
│       └── provider[registry.terraform.io/hashicorp/null] >= 2.1.0
│   ├── module.kms
│       └── provider[registry.terraform.io/hashicorp/google] >= 3.53.0, < 6.0.0
│   └── module.seed_project
│       ├── provider[registry.terraform.io/hashicorp/google-beta] >= 4.28.0, < 6.0.0
│       ├── provider[registry.terraform.io/hashicorp/google] >= 4.28.0, < 6.0.0
│       ├── module.project-factory
│           ├── provider[registry.terraform.io/hashicorp/random] >= 2.2.0
│           ├── provider[registry.terraform.io/hashicorp/time] >= 0.5.0
│           ├── provider[registry.terraform.io/hashicorp/google] >= 3.50.0, < 6.0.0
│           ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.50.0, < 6.0.0
│           ├── provider[registry.terraform.io/hashicorp/null] >= 2.1.0
│           └── module.project_services
│               ├── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 6.0.0
│               └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.43.0, < 6.0.0
│       ├── module.quotas
│           └── provider[registry.terraform.io/hashicorp/google-beta] >= 4.11.0, < 6.0.0
│       ├── module.shared_vpc_access
│           ├── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 6.0.0
│           └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.43.0, < 6.0.0
│       ├── module.budget
│           └── provider[registry.terraform.io/hashicorp/google] >= 4.28.0, < 6.0.0
│       ├── module.essential_contacts
│           ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.43.0, < 6.0.0
│           └── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 6.0.0
│       └── module.gsuite_group
│           └── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 6.0.0
├── module.seed_project_iam_member
│   ├── provider[registry.terraform.io/hashicorp/random]
│   ├── provider[registry.terraform.io/hashicorp/google] >= 3.77.0
│   └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.77.0
├── module.optional_group
│   ├── provider[registry.terraform.io/hashicorp/google] >= 3.67.0, < 6.0.0
│   └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.67.0, < 6.0.0
├── module.tfc-oidc
│   └── provider[registry.terraform.io/hashicorp/google] >= 3.53.0, < 5.0.0
├── module.parent_iam_member
│   ├── provider[registry.terraform.io/hashicorp/random]
│   ├── provider[registry.terraform.io/hashicorp/google] >= 3.77.0
│   └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.77.0
├── module.cicd_project_iam_member
│   ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.77.0
│   ├── provider[registry.terraform.io/hashicorp/random]
│   └── provider[registry.terraform.io/hashicorp/google] >= 3.77.0
├── module.tfc_agent_gke
│   ├── provider[registry.terraform.io/hashicorp/random] >= 3.4.3, < 4.0.0
│   ├── provider[registry.terraform.io/hashicorp/google] >= 4.3.0, < 6.0.0
│   ├── provider[registry.terraform.io/hashicorp/kubernetes] >= 2.0.0, < 3.0.0
│   ├── provider[registry.terraform.io/hashicorp/google-beta]
│   ├── module.private_service_connect
│       ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.50.0, < 6.0.0
│       ├── provider[registry.terraform.io/hashicorp/google] >= 3.50.0, < 6.0.0
│       ├── module.googleapis
│           ├── provider[registry.terraform.io/hashicorp/google] >= 4.40.0, < 6.0.0
│           └── provider[registry.terraform.io/hashicorp/google-beta] >= 4.40.0, < 6.0.0
│       ├── module.pkg_dev
│           ├── provider[registry.terraform.io/hashicorp/google-beta] >= 4.40.0, < 6.0.0
│           └── provider[registry.terraform.io/hashicorp/google] >= 4.40.0, < 6.0.0
│       └── module.gcr
│           ├── provider[registry.terraform.io/hashicorp/google] >= 4.40.0, < 6.0.0
│           └── provider[registry.terraform.io/hashicorp/google-beta] >= 4.40.0, < 6.0.0
│   ├── module.tfc_agent_cluster
│       ├── provider[registry.terraform.io/hashicorp/kubernetes] ~> 2.10
│       ├── provider[registry.terraform.io/hashicorp/random] >= 2.1.0
│       ├── provider[registry.terraform.io/hashicorp/google] >= 5.9.0, < 6.0.0
│       └── provider[registry.terraform.io/hashicorp/google-beta] >= 5.9.0, < 6.0.0
│   ├── module.hub
│       ├── provider[registry.terraform.io/hashicorp/google] >= 5.6.0, < 6.0.0
│       └── provider[registry.terraform.io/hashicorp/google-beta] >= 5.6.0, < 6.0.0
│   └── module.network
│       ├── provider[registry.terraform.io/hashicorp/google-beta] >= 4.64.0, < 6.0.0
│       ├── provider[registry.terraform.io/hashicorp/google] >= 4.64.0, < 6.0.0
│       ├── module.routes
│           └── provider[registry.terraform.io/hashicorp/google] >= 3.83.0, < 6.0.0
│       ├── module.subnets
│           └── provider[registry.terraform.io/hashicorp/google] >= 4.25.0, < 6.0.0
│       ├── module.vpc
│           ├── provider[registry.terraform.io/hashicorp/google-beta] >= 4.64.0, < 6.0.0
│           └── provider[registry.terraform.io/hashicorp/google] >= 4.64.0, < 6.0.0
│       └── module.firewall_rules
│           └── provider[registry.terraform.io/hashicorp/google] >= 3.33.0, < 6.0.0
├── module.org_iam_member
│   ├── provider[registry.terraform.io/hashicorp/google] >= 3.77.0
│   ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.77.0
│   └── provider[registry.terraform.io/hashicorp/random]
├── module.tfc_cicd
│   ├── provider[registry.terraform.io/hashicorp/google-beta] ~> 4.5
│   ├── provider[registry.terraform.io/hashicorp/google] ~> 4.5
│   ├── module.project-factory
│       ├── provider[registry.terraform.io/hashicorp/time]
│       ├── provider[registry.terraform.io/hashicorp/google] >= 3.50.0, < 5.0.0
│       ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.50.0, < 5.0.0
│       ├── provider[registry.terraform.io/hashicorp/null] >= 2.1.0
│       ├── provider[registry.terraform.io/hashicorp/random] >= 2.2.0
│       └── module.project_services
│           ├── provider[registry.terraform.io/hashicorp/google-beta] >= 3.43.0, < 5.0.0
│           └── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 5.0.0
│   ├── module.quotas
│       └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.1.0, < 5.0.0
│   ├── module.shared_vpc_access
│       ├── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 5.0.0
│       └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.43.0, < 5.0.0
│   ├── module.budget
│       └── provider[registry.terraform.io/hashicorp/google] ~> 4.5
│   ├── module.essential_contacts
│       ├── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 5.0.0
│       └── provider[registry.terraform.io/hashicorp/google-beta] >= 3.43.0, < 5.0.0
│   └── module.gsuite_group
│       └── provider[registry.terraform.io/hashicorp/google] >= 3.43.0, < 5.0.0
└── module.bootstrap_projects_remove_editor
    └── provider[registry.terraform.io/hashicorp/google] >= 3.77.0

 ~/src/github/terraform-google-modules/gcp-bootstrap/envs/shared  plan ?1 

danscarf commented 6 months ago

The dependency conflict is from the tfc_agent_gke module.

This depends on google-beta versions >= 5.6.0, which creates a conflict with several other dependencies of this module which constrain the version below 5.0.0.

My workaround was to comment out the module definition, as the init pulls down the modules even if enable_tfc_cloud_agents = false.

# module "tfc_agent_gke" {
#   source = "./modules/tfc-agent-gke"
#   count  = var.enable_tfc_cloud_agents == true ? 1 : 0

#   project_id             = module.tfc_cicd.project_id
#   project_number         = module.tfc_cicd.project_number
#   tfc_agent_token        = tfe_agent_token.tfc_agent_token[0].token
#   create_service_account = false
#   service_account_email  = google_service_account.terraform-env-sa["bootstrap"].email
#   service_account_id     = google_service_account.terraform-env-sa["bootstrap"].id

#   //If you are using Terraform Cloud Agents, un-comment this block after the first apply according README instructions
#   # providers = {
#   #   kubernetes = kubernetes
#   # }
# }

Also, there hasn't been any activity on this issue for about a week. Did I miss a step in filing the issue? Thank you!

github-actions[bot] commented 4 months ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days

daniel-cit commented 3 months ago

depends of https://github.com/GoogleCloudPlatform/terraform-google-tf-cloud-agents/pull/24

eeaton commented 2 months ago

@apeabody Can you review and merge https://github.com/GoogleCloudPlatform/terraform-google-tf-cloud-agents/pull/24 ? We should be able to close this issue once that is complete

terraform-google-modules / terraform-example-foundation