1-org tf plan check step 9 requires a 0-bootstrap plan check on backend validation errors - due to terraform 1.3.0 (docker) and 1.7.4 (console) mismatch - may require console terraform downgrade before starting deployment and creating the state file

obriensystems commented 5 months ago

TL;DR

step 9 of https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/1-org/README.md#deploying-with-cloud-build

1-org - step 9 checking tf plan issues - was due to bootstrap

gcp-org - plan

**************************************************
data.terraform_remote_state.bootstrap: Reading...
module.cai_monitoring.data.archive_file.function_source_zip: Reading...
module.cai_monitoring.data.archive_file.function_source_zip: Read complete after 0s [id=1e9314009f01646867d2cae991af75d380d72df9]
module.org_domain_restricted_sharing.data.google_organization.orgs["obrienlabs.app"]: Reading...
module.org_domain_restricted_sharing.data.google_organization.orgs["obrienlabs.app"]: Read complete after 0s [id=organizations/630259462753]

Error: Error loading state error

  with data.terraform_remote_state.bootstrap,
  on remote.tf line 38, in data "terraform_remote_state" "bootstrap":
  38:   backend = "gcs"

error loading the remote state: 16 problems:

- unsupported checkable object kind "var"

same issue for gcp-bootstrap

Initializing the backend...

Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.
Error loading state: 16 problems:

- unsupported checkable object kind "var"

Expected behavior

No response

Observed behavior

No response

Terraform Configuration

1.7.4 in console 1.3.0 in docker

Terraform Version

1.7.4 in console 1.3.0 in docker

Additional information

No response

obriensystems commented 5 months ago

1-org tf plan check step 9 requires a 0-bootstrap plan check on backend validation errors - due to terraform 1.3.0 (docker) and 1.7.4 (console) mismatch - may require console terraform downgrade before starting deployment before creating the state file

fmichaelobrien commented 5 months ago

see step 9 in main jira https://github.com/terraform-google-modules/terraform-example-foundation/issues/1133#issuecomment-1991845216

fmichaelobrien commented 5 months ago

1-org step 9 terraform version mismatch mitigation

I was hoping to avoid a terraform downgrade until we get into refactoring but it looks like a 1.7.4 to 1.3.0 TF downgrade is required before creating the state file - or we get into a situation where the cloud build triggered plan running 1.3.0 cannot reconcile with the state file created in the gcp console under 1.7.4. Either we restart the deployment after downgrading or upgrade the TEF to TF 1.7 - will need to do this before moving on to 2-environments

obriensystems commented 5 months ago

TF Versions Cloud Build via Dockerfile is 1.3.0 https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/Dockerfile#L18

ARG TERRAFORM_VERSION=1.3.0

GCP console as of 202403 is 1.7.4

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ terraform --version
Terraform v1.7.4
on linux_amd64

obriensystems commented 5 months ago

for change tracking post terraform 1.3.0 follow https://github.com/terraform-google-modules/terraform-example-foundation/pull/831 https://github.com/terraform-google-modules/terraform-example-foundation/pull/1003/files

obriensystems commented 5 months ago

Update: should have followed the warning around 1.3.0 in https://github.com/terraform-google-modules/terraform-example-foundation/pull/831/files https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/README.md#prerequisites

Note: Make sure that you use version 1.3.0 of Terraform throughout this series. Otherwise, you might experience Terraform state snapshot lock errors.

obriensystems commented 5 months ago

Terraform local downgrade procedure - OSX (watch the sh line endings) - do windows 11 as well

get 1.3 https://releases.hashicorp.com/terraform/ https://releases.hashicorp.com/terraform/1.3.0/ https://releases.hashicorp.com/terraform/1.3.0/terraform_1.3.0_darwin_arm64.zip

drop terraform binary into a path folder
michaelobrien@mbp7 TEF-GCP-LZ-HS % terraform --version
Terraform v1.3.0
on darwin_arm64

Your version of Terraform is out of date! The latest version
is 1.7.4. You can update by downloading from https://www.terraform.io/downloads.html

obriensystems commented 5 months ago

The validation script needs to be updated https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/scripts/validate-requirements.sh#L94

TF_VERSION="1.3.0"
...
        TERRAFORM_CURRENT_VERSION=$(terraform version -json | jq -r .terraform_version)
        if [ "$(compare_version "$TERRAFORM_CURRENT_VERSION" "$TF_VERSION")" -gt 1 ]; then
            echo_wrong_version "Terraform" "greater than or equal to" "$TF_VERSION" "https://learn.hashicorp.com/tutorials/terraform/install-cli" "$TERRAFORM_CURRENT_VERSION"
            ERRORS+=$'  Terraform version is incompatible.\n'
        fi

change

-greater than or equal to
+equal to

-        if [ "$(compare_version "$TERRAFORM_CURRENT_VERSION" "$TF_VERSION")" -gt 1 ]; then
 +       if [ "$(compare_version "$TERRAFORM_CURRENT_VERSION" "$TF_VERSION")" -eq 1 ]; then
## test this change first (locally in just a script unit test or in a new-org deploy)

Note we can go to 1.3.7 - on a minor version but 1.4+ is untested

fmichaelobrien commented 4 months ago

stale bot timer restart - https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/.github/workflows/stale.yml#L21

terraform-google-modules / terraform-example-foundation