Atlantis still doesn't support Terraform 1.7.x

tibuntu commented 6 months ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Overview of the Issue

Even though these changes: https://github.com/runatlantis/atlantis/pull/4203 made it into the most recent version v0.27.2 Atlantis is still not able to handle Terraform version 1.7.x

Reproduction Steps

Have a versions.tf like:

terraform {
  required_version = "~> 1.7"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.40"
    }
  }
}

then simply run atlantis plan (See logs below)

Logs

running "terraform init -upgrade -input=false" in "/atlantis-data/repos/<our-repo>/241/default/environments/test": exit status 1: running "terraform init -upgrade -input=false" in "/atlantis-data/repos/<our-repo>/241/default/environments/test": 

Initializing the backend...

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Upgrading modules...
- kms in ../../modules/<our-module>
╷
│ Error: Unsupported Terraform Core version
│ 
│   on providers.tf line 11, in terraform:
│   11:   required_version = "~> 1.7"
│ 
│ This configuration does not support Terraform version 1.6.3. To proceed,
│ either choose another supported Terraform version or update this version
│ constraint. Version constraints are normally set for good reason, so
│ updating the constraint may lead to other errors or unexpected behavior.
╵

Environment details

Atlantis version: v0.27.2
Deployment method: helm

Atlantis server-side config file:

        - name: ATLANTIS_REPO_ALLOWLIST
          value: '*'
        - name: ATLANTIS_WRITE_GIT_CREDS
          value: 'true'
        - name: ATLANTIS_API_SECRET
          valueFrom:
            secretKeyRef:
              key: ATLANTIS_API_SECRET
              name: atlantis
        - name: ATLANTIS_GH_APP_KEY
          valueFrom:
            secretKeyRef:
              key: ATLANTIS_GH_APP_KEY
              name: atlantis
        - name: ATLANTIS_GH_WEBHOOK_SECRET
          valueFrom:
            secretKeyRef:
              key: ATLANTIS_GH_WEBHOOK_SECRET
              name: atlantis
        - name: ATLANTIS_DATA_DIR
          value: /atlantis-data
        - name: ATLANTIS_PORT
          value: '4141'
        - name: ATLANTIS_REPO_CONFIG
          value: /etc/atlantis/repos.yaml

Repo atlantis.yaml file:

version: 3
automerge: true
projects:
- name: test
  dir: environments/test
  autoplan:
    enabled: false

- name: stage
  dir: environments/stage
  autoplan:
    enabled: false

- name: prod
  dir: environments/prod
  autoplan:
    enabled: false

workflows:
  default_workflow:
    plan:
      steps:
      - run: terraform init -upgrade -input=false
      - run: terraform plan -input=false -refresh -out $PLANFILE
    apply:
      steps:
      - run: terraform apply $PLANFILE

tibuntu commented 6 months ago

Did it actually made it into the release?

I had a quick look inside the container:

> docker run -ti --entrypoint /bin/sh ghcr.io/runatlantis/atlantis:v0.27.2
/ $ which terraform
/usr/local/bin/terraform
/ $ terraform --version
Terraform v1.6.3
on linux_amd64

Your version of Terraform is out of date! The latest version
is 1.7.5. You can update by downloading from https://www.terraform.io/downloads.html

paulbailey commented 5 months ago

That's strange, we have it working for Terraform 1.7...

From the docs:

The highest version of Terraform allowed in your code is the version specified by DEFAULT_TERRAFORM_VERSION in the image your server is running.

Is that perhaps set somewhere on the image you're running?

stasostrovskyi commented 5 months ago

If we check a commit from which the latest release was built, there is still a 1.6 version of Terraform being used - https://github.com/runatlantis/atlantis/blob/v0.27.2/Dockerfile#L6. That said, I think the simplest thing is to have your own image based on atlantis one and update terraform at your own pace. You can copy terraform installation from atlantis image for example.

paulbailey commented 5 months ago

But if you run the stock image, with that environment variable set to a higher version, it will download the requested version. That's the behaviour I see on our implementation.

stasostrovskyi commented 5 months ago

That is true, but only for default workflow. If you are using custom workflows there is no auto-download. Which in itself may as well be a bug :)

peikk0 commented 5 months ago

You need to run /atlantis-data/bin/terraform${ATLANTIS_TERRAFORM_VERSION} and not just terraform (which is the one baked into the image) in your custom workflow.

tibuntu commented 5 months ago

Hello everyone, thank you for all the feedback!

@stasostrovskyi that was a very good hint, and I agree that this might be something to debate :)

Thanks to the input from @peikk0, I was able to adapt our custom workflow:

workflows:
  default_workflow:
    plan:
      steps:
      - run: /atlantis-data/bin/terraform${ATLANTIS_TERRAFORM_VERSION} init -upgrade -input=false
      - run: /atlantis-data/bin/terraform${ATLANTIS_TERRAFORM_VERSION} plan -input=false -refresh -out $PLANFILE
    apply:
      steps:
      - run: /atlantis-data/bin/terraform${ATLANTIS_TERRAFORM_VERSION} apply $PLANFILE

and adapted our Helm values to use a specific version that we want to use by providing defaultTFVersion: 1.7.4

Now we can control which version to use via Helm, and Atlantis correctly downloads the desired version!

@paulbailey @nitrocode any more remarks or inputs? From my perspective this can be closed.

nitrocode commented 5 months ago

That's how I also run terraform for what it's worth.

I've always been able to run every terraform release using the autodownload via required_version and the workflow using ATLANTIS_TERRAFORM_VERSION.

It might be worth changing this ticket to Document how to use the version in "required_version" if it's not already documented.

ricardosilva86 commented 5 months ago

Having to use terraform${ATLANTIS_TERRAFORM_VERSION} is a bit odd. For my use case, we need a custom workflow because we run only one instance, but we deploy in 2 different GCP projects, which requires 2 different Service Accounts. Then we use:

    plan:
      steps:
      - env:
          name: GOOGLE_IMPERSONATE_SERVICE_ACCOUNT
          value: <service_account_email>

We never had issues with different TF versions before, but since last update, it started to fail with the same error @tibuntu reported.

│ Error: Unsupported Terraform Core version
│ 
│   on versions.tf line 2, in terraform:
│    2:   required_version = "=1.7.4"
│ 
│ This configuration does not support Terraform version 1.6.3. To proceed,
│ either choose another supported Terraform version or update this version
│ constraint. Version constraints are normally set for good reason, so
│ updating the constraint may lead to other errors or unexpected behavior.

And I really wouldn't like to have to set this terraform${ATLANTIS_TERRAFORM_VERSION}. Any other workaround that could prevent us from having to set this atlantis tf version variable?

nitrocode commented 5 months ago

@ricardosilva86 are you saying that 0.27.2 has a regression and 0.27.1 worked as expected without using terraform${ATLANTIS_TERRAFORM_VERSION} in your workflow ?

runatlantis / atlantis