renovatebot / renovate

Home of the Renovate CLI: Cross-platform Dependency Automation by Mend.io
https://mend.io/renovate
GNU Affero General Public License v3.0
17.15k stars 2.23k forks source link

Upgrade Terraform version in Atlantis projects #18847

Open secustor opened 1 year ago

secustor commented 1 year ago

What would you like Renovate to be able to do?

Atlantis is a deployment tool which is used to apply Terraform IaC.

It provides a option to define repository level configuration in a defined file ( atlantis.yml ).
Here the to be used Terraform version can be defined.

version: 3
automerge: false
projects:
  - name: my-project
    dir: ./my-project
    terraform_version: v1.1.5
    autoplan:
      when_modified:
        - "*.tf"
        - "./**/*.tf*"
      enabled: true
  - name: my-other-project
    dir: ./my-other-project
    terraform_version: v1.2.0
    autoplan:
      when_modified:
        - "*.tf"
        - "./**/*.tf*"
      enabled: true

If you have any ideas on how this should be implemented, please tell us here.

For this a new manager is needed to parse the file and extract the Terraform version.
The datasource and most of the implementation can be reused from the terraform-version manager.

Is this a feature you are interested in implementing yourself?

Maybe

github-actions[bot] commented 1 year ago

Hi there,

You're asking us to support a new package manager. We need to know some basic information about this package manager first. Please copy/paste the new package manager questionnaire, and fill it out in full.

Once the questionnaire is filled out we'll decide if we want to support this new manager.

Good luck,

The Renovate team

viceice commented 1 year ago

it seems for now you can a regex manager to workaround.

secustor commented 1 year ago

True, this is for reference and gauging if there is enough interest to facilitate an own pre-setup manager.

secustor commented 1 year ago

New package manager questionnaire

Did you read our documentation on adding a package manager?

Basics

Name of package manager

Atlantis

What language does this support?

Terraform

How popular is this package manager?

Widely used in big enterprises

Does this language have other (competing?) package managers?


Package File Detection

What type of package files and names does it use?

It uses YAML files which are located at the repository root and have to named atlantis.yaml

What fileMatch pattern(s) should be used?

atlantis.ya?ml

Is it likely that many users would need to extend this pattern for custom file names?

Is the fileMatch pattern likely to get many "false hits" for files that have nothing to do with package management?

No, false hits should be very scarce or not happening at all

Parsing and Extraction

Can package files have "local" links to each other that need to be resolved?

No

Is there a reason why package files need to be parsed together (in serial) instead of independently?

No

What format/syntax is the package file in?

How do you suggest parsing the file?

Does the package file structure distinguish between different "types" of dependencies? e.g. production dependencies, development dependencies, etc?

List all the sources/syntaxes of dependencies that can be extracted

As of now there is the only the to be used Terraform version

Describe which types of dependencies above are supported and which will be implemented in future


Versioning

What versioning scheme does the package file(s) use?

Terraform uses semantic versioning

Does this versioning scheme support range constraints, e.g. ^1.0.0 or 1.x?


Lookup

Is a new datasource required? Provide details

Will users need the capability to specify a custom host/registry to look up? Can it be found within the package files, or within other files inside the repository, or would it require Renovate configuration?

No, need for this.

Do the package files have any "constraints" on the parent language (e.g. supports only v3.x of Python) or platform (Linux, Windows, etc.) that should be used in the lookup procedure?

No

Will users need the ability to configure language or other constraints using Renovate config?

Shouldn't be necessary

Artifacts

Are lock files or checksum files used? Are they mandatory?

No, there are not lock files

If so, what tool and exact commands should be used if updating one or more package versions in a dependency file?

If applicable, describe how the tool maintains a cache and if it can be controlled via CLI or environment variables? Do you recommend the cache be kept or disabled/ignored?

If applicable, what command should be used to generate a lock file from scratch if you already have a package file? This will be used for "lock file maintenance"

Other

Is there anything else to know about this package manager?

JoshiiSinfield commented 1 year ago

We use atlantis quite heavily here at CDL. We don't specify the terraform_version unless strictly required to pin it to an older version.

We simply use the TF versions shipped with Atlantis, and keep Atlantis updated as regularly as possible.

Can I ask what your use case is? Are you specifying terraform_version / wantint Renovate to update it to get the latest and greatest TF every time - newer than what is shipped with Atlantis?

Cheers, Josh

secustor commented 1 year ago

We add the terraform_version to check for diffs on upgrades. The workflow in this case would be that Renovate updates the terraform_version, the plans are checked for any diffs and then merged.
This is to not potentially poison the PRs with diffs which are originating from non-transparent code changes.

mkurde commented 1 year ago

it seems for now you can a regex manager to workaround.

I try to make it work with the reg-ex manager but it seems I am to new to the syntax. I struggle at the datasource. @secustor did you manage to setup the regex manager and if yes, could your share the config snippet?

We also make use of the terraform_version in the atlantis config as every time might use a different terraform version. In case of autoplan together with a updated required terraform core version via dependabot the workflow will fail. so we need to update the atlantis.yaml as well to keep that in sync.

secustor commented 1 year ago

@mkurde We are using github-releases as datasource and depName with hashicorp/terraform as value in our terraform-version manager.

mkurde commented 1 year ago

I got it working, but its not going to work for us.

"regexManagers": [
    {
      "fileMatch": ["atlantis.yaml"],
      "matchStrings": ["terraform_version: '(v?<currentValue>.*)'"],
      "depNameTemplate": "hashicorp/terraform",
      "datasourceTemplate": "github-releases",
      "versioningTemplate": "hashicorp"
    }
  ]

As we are grouping the terraform changes into one PR ...

"packageRules": [
    {
      "matchManagers": [
        "terraform",
        "terraform-version"
      ],
      "groupName": "Terraform"
    },

... we have to also group in the atlantis.yaml change within the same PR on order to allow atlantis's autoplan work whenever the required terraform version was updated.

I am thinking about filling this issue here and adding an atlantis manager to be able to add him to the terraform group above.

viceice commented 1 year ago

use matchPackageName to group. is much easier and works with regex manager. 😉