search

binbashar / le-tf-infra-aws

Terraform code for Leverage Reference Architecture for AWS, designed under optimal configs for the most popular modern web and mobile applications needs.
https://www.binbash.co/leverage
Apache License 2.0
27 stars 8 forks source link

Enhancement | Check and Update layers to use terraform, modules and provider resources in the latest versions #474

Open rodriguez-matias opened 1 year ago

rodriguez-matias commented 1 year ago

What?

• Keep Updated all Terraform config on every layer.
• Keep all versions update changes registered in one place.

How?

• Check and Update versions of Terraform Core, Providers, and Modules.
• Get the latest release version from Terraform Registry.
• Update version constraints.
• Test layers with new versions and report potentials issues and parameters changes.

Why?

• Keeping Leverage Reference Architecture up to date.

Versions to consider for updates:

leverage cli: "v1.9.2"

terraform {
  required_version = "~> 1.3.5"

  required_providers {
    aws        = "~> 4.10"
    kubernetes = "~> 2.10"
    helm       = "~> 2.5"
    vault      = "~> 3.6"  
}

ChangeLog

Current Version Upgrade Status

├── apps-devstg │ ├── global │ │ └── base-identities ✅ (tf > 1.x / tf-aws > 4.x) │ │ └── cli-test-layer ✅ (tf > 1.x / tf-aws > 4.x) │ ├── us-east-1 │ │ ├── backups\ -- ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── base-certificates ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── base-network ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── base-tf-backend ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── cdn-s3-frontend\ -- ✅ (tf > 1.x / tf-aws = 3.x) │ │ ├── databases-aurora ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── databases-mysql\ -- ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── databases-pgsql\ -- │ │ ├── ec2-fleet-ansible\ -- ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── k8s-eks │ │ │ ├── cluster ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ │ ├── identities ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ │ ├── k8s-resources. ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ │ ├── k8s-workloads. ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ │ └── network. ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ ├── k8s-eks-demoapps │ │ │ ├── cluster ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ │ ├── identities ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ │ ├── k8s-resources. ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ │ ├── k8s-workloads. ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ │ └── network. ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ ├── k8s-eks-v1.17 │ │ │ ├── cluster ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ │ ├── identities ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ │ ├── k8s-resources. ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ │ ├── k8s-workloads. ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ │ └── network. ✅ (tf > 1.x / tf-aws > 4.x / tf-k8s > 2.x) │ │ ├── k8s-kind │ │ ├── k8s-kops\ -- │ │ ├── notifications ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-audit ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-base ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-certs ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-compliance\ -- │ │ ├── security-firewall\ -- ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-keys ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── storage │   │ │ └── s3-bucket-demo-files. ✅ (tf > 1.x / tf-aws > 4.x) │ │ └── tools-cloud-nuke ✅ (tf > 1.x / tf-aws > 4.x) │ └── us-east-2 │ ├── k8s-eks-v.1.17 │ ├── security-compliance\ -- │ └── security-keys ✅ (tf > 1.x / tf-aws > 4.x) ├── apps-prd │ ├── global │ │ └── base-identities ✅ (tf > 1.x / tf-aws > 4.x) │ └── us-east-1 │ ├── backups\ -- ✅ (tf > 1.x / tf-aws > 4.x) │ ├── base-network ✅ (tf > 1.x / tf-aws > 4.x) │ ├── base-tf-backend ✅ (tf > 1.x / tf-aws > 4.x) │ ├── cdn-s3-frontend\ -- ✅ (tf > 1.x / tf-aws = 3.x) │ ├── ec2-fleet\ -- ✅ (tf > 1.x / tf-aws > 4.x) │ ├── notifications ✅ (tf > 1.x / tf-aws > 4.x) │ ├── security-audit ✅ (tf > 1.x / tf-aws > 4.x) │ ├── security-base ✅ (tf > 1.x / tf-aws > 4.x) │ ├── security-certs ✅ (tf > 1.x/ tf-aws > 4.x) │ ├── security-compliance\ -- │ └── security-keys ✅ (tf > 1.x / tf-aws > 4.x)
├── management │ ├── global │ │ ├── base-identities ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── cost-mgmt │ │ ├── organizations │ │ └── sso ✅ (tf > 1.x / tf-aws > 4.x) │ ├── us-east-1 │ │ ├── backups │ │ ├── base-tf-backend ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── firewall-manager │ │ ├── notifications ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-audit ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-base ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-compliance │ │ ├── security-keys ✅ (tf > 1.x / tf-aws > 4.x) │ │ └── security-monitoring ✅ (tf > 1.x/ tf-aws > 4.x) │ └── us-east-2 │ └── security-monitoring\ -- ✅ (tf > 1.x/ tf-aws > 4.x) ├── network │ ├── global │ │ └── base-identities ✅ (tf > 1.x / tf-aws > 4.x) │ ├── us-east-1 │ │ ├── base-network ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── base-tf-backend ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── network-firewall │ │ ├── notifications ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-audit ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-base ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-compliance\ -- │ │ ├── security-keys ✅ (tf > 1.x / tf-aws > 4.x) │ │ └── transit-gateway │ └── us-east-2 │ ├── base-network ✅ (tf > 1.x / tf-aws > 4.x) │ ├── network-firewall │ ├── security-compliance\ -- │ ├── security-keys ✅ (tf > 1.x / tf-aws > 4.x) │ └── transit-gateway

├── security │ ├── global │ │ └── base-identities ✅ (tf > 1.x / tf-aws > 4.x) │ ├── us-east-1 │ │ ├── base-tf-backend ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── firewall-manager │ │ ├── notifications ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-audit ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-base ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── security-compliance\ -- │ │ ├── security-keys ✅ (tf > 1.x / tf-aws > 4.x) │ │ └── security-monitoring ✅ (tf > 1.x/ tf-aws > 4.x) │ └── us-east-2 │ ├── security-audit │ ├── security-compliance\ -- │ └── security-monitoring\ -- ✅ (tf > 1.x/ tf-aws > 4.x) | └── shared ├── global │ ├── base-dns ✅ (tf > 1.x / tf-aws > 4.x) │ └── base-identities ✅ (tf > 1.x / tf-aws > 4.x) ├── us-east-1 │ ├── backups ✅ (tf > 1.x / tf-aws > 4.x) │ ├── base-network ✅ (tf > 1.x / tf-aws > 4.x) │ ├── base-tf-backend ✅ (tf > 1.x / tf-aws > 4.x) │ ├── container-registry ✅ (tf > 1.3.x / tf-aws > 4.10) │ ├── ec2-fleet\ -- ✅ (tf > 1.x / tf-aws > 4.x) │ ├── ec2-fleet-bastions\ -- │ ├── k8s-eks │ ├── k8s-eks-demoapps │ ├── k8s-eks-prd │ ├── notifications ✅ (tf > 1.x / tf-aws > 4.x) │ ├── secrets-manager\ -- │ ├── security-audit ✅ (tf > 1.x / tf-aws > 4.x) │ ├── security-base ✅ (tf > 1.x / tf-aws > 4.x) │ ├── security-compliance\ -- │ ├── security-keys ✅ (tf > 1.x / tf-aws > 4.x) │ ├── storage │ │ ├── backup-gdrive ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── object-file-shares-for-users-list ✅ (tf > 1.x / tf-aws > 4.x) │ │ ├── object-file-shares-for-sftp ✅ (tf > 1.x / tf-aws > 4.x) │ ├── tools-cloud-scheduler-stop-start │ ├── tools-eskibana │ ├── tools-github-selfhosted-runners │ ├── tools-jenkins\ -- │ ├── tools-managedeskibana │ ├── tools-prometheus │ ├── tools-vault │ ├── tools-vpn-server ✅ (tf > 1.x / tf-aws > 4.x) │ └── tools-webhooks\ -- └── us-east-2 ├── base-network ✅ (tf > 1.x / tf-aws > 4.x) ├── container-registry ✅ (tf > 1.3.x / tf-aws > 4.10) ├── security-compliance\ -- ├── security-keys ✅ (tf > 1.x / tf-aws > 4.x) ├── tools-eskibana └── tools-prometheus

Ref Links

rodriguez-matias commented 1 year ago

Layer le-tf-infra-aws/management/global/sso updated by this PR https://github.com/binbashar/le-tf-infra-aws/pull/481

rodriguez-matias commented 1 year ago

Hey, I found this custom Github Action workflow that is a wrapper for the tfupdate command:

tfupdate-action https://github.com/HENNGE/tfupdate-action

I'll be testing this workflow to try to automate the steps for update the layers.

exequielrafaela commented 1 year ago

@rodriguez-matias may be as part of this ongoing layer effort we can start integrating default tags in the terraform aws provider as best practice. Ideally having a dedicated issue for this could help segmenting the scope for this task and being able to prioritize accordingly. So If you have a few mins and can create the issue I really appreciate it. I think it should look somthing similar too:

image

One consideration I haven't tested yet is the possibility to parametrize this default tags and pass it arguments from our common configs tfvars files.

CC: @binbashar/leverage-ref-architecture-aws-admin @binbashar/leverage-ref-architecture-aws-dev

exequielrafaela commented 7 months ago

This new leverage cli feature request https://github.com/binbashar/leverage/issues/259 should help with this.

exequielrafaela commented 5 months ago

@lgallard Let's discuss the upgrade of the following layers:

@diego-ojeda-binbash Let's review and adjust this versions accordingly: 

leverage cli: "v1.9.2"

terraform {
  required_version = "~> 1.3.5"

  required_providers {
    aws        = "~> 4.10"
    kubernetes = "~> 2.10"
    helm       = "~> 2.5"
    vault      = "~> 3.6"  
}
diego-ojeda-binbash commented 5 months ago

@exequielrafaela @lgallard Sure, here:

leverage cli: "v1.12.2"     => latest available if possible

terraform {
  required_version = "~> 1.6"      => latest available if possible, if it fails due to constraints we can move to 1.5

  required_providers {
    aws        = "~> 5.0"
    kubernetes = "~> 2.10"            => latest available if possible, must be tested by standing up the cluster
    helm       = "~> 2.5"             => latest available if possible, must be tested by standing up the cluster
    vault      = "~> 3.6"             => we don't use this any more
}

And keep in mind you can spin up the demoapps cluster using these instructions: https://binbash.atlassian.net/wiki/spaces/BDPS/pages/2270527489/DemoApps#Standing-up-the-DemoApps