jetpackets / cloud-handbook

MIT License
0 stars 0 forks source link

As a DevOps Engineer I want to easily initialize Terraform's backend at AWS to store the state remotely #1

Open vflopes opened 2 years ago

vflopes commented 2 years ago

There're some approaches to this implementation:

  1. Terraform only
  2. Terraform and custom provider (Golang)
  3. Terragrunt and Terraform
  4. Ansible and Terraform
  5. Ansible only
  6. Custom application (Golang)

Terraform only

Complexity: low Match Requirements: yes Operational Scalability: high

Terraform and custom provider (Golang)

Complexity: high (need to publish and mantain a provider) Match Requirements: yes Operational Scalability: medium (need a team with Golang skills)

Terragrunt and Terraform

Complexity: medium (need to learn one more tool, not so popular) Match Requirements: maybe (Terragrunt is not in our toolset) Operational Scalability: high

Ansible and Terraform

Complexity: high (two completely different tools) Match Requirements: maybe (we don't have Ansible in our toolset) Operational Scalability: medium (although Ansible is a popular tool, this option requires a skill with it from our teams)

Ansible only

Complexity: low Match Requirements: maybe (Ansible is not so good as Terraform to provision infrastructure) Operational Scalability: high

Custom application (Golang)

Complexity: high (need to maintain an application) Match Requirements: yes Operational Scalability: low (need a highly skilled team)

codermarcos commented 2 years ago

I would like contribute with a little experience that i had on the first scenario using only terraform!I tried to implement this using the following logic:

To first run:

  1. I use the data sources to verify if the module already exists to no try create. like this:
data "azurerm_resources" "exists" {
  resource_group_name = local.rg_name
  type                = "Microsoft.Storage/storageAccounts"
}
  1. I create a bucket to save the state if the bucket no exists

    
    resource "azurerm_resource_group" "main" {
    count    = length(data.azurerm_resources.exists.resources) == 0 ? 1 : 0
    name     = local.rg_name
    location = var.resource_group_location
    
    depends_on = [
    data.azurerm_resources.exists
    ]
    }

resource "azurerm_storage_account" "main" { count = local.create ? 1 : 0

name = local.name_alphanum resource_group_name = azurerm_resource_group.main[0].name location = azurerm_resource_group.main[0].location

account_kind = "BlobStorage" min_tls_version = "TLS1_2" enable_https_traffic_only = true is_hns_enabled = false nfsv3_enabled = false shared_access_key_enabled = var.shared_access_key_enabled allow_blob_public_access = false

account_tier = var.account_tier account_replication_type = var.account_replication_type

depends_on = [ data.azurerm_resources.exists ] }

resource "azurerm_storage_container" "main" { count = local.create ? 1 : 0

name = local.container_name storage_account_name = azurerm_storage_account.main[0].name container_access_type = "private"

depends_on = [ data.azurerm_resources.exists ] }


3. Then run my terraform to create infrastructure

⚠️**Problem**

This has a big problem, terraform is declarative, that is he should decide when create or no create.

When I **first** tried this I had the following problem:

* ✅ In the first run it ok. Terraform create the bucket 
* ❌ In the second run fails. Because terraform see that already exists this resource then no create the bucket but the problem is he created then he understands that should destroy since the resource should no longer exist.

♻️ **Solution**

To solve this i use this lifecycle in all resources that has been created:

```hcl
  lifecycle {
    ignore_changes = all
  }

So next time it won't destroy existing resources. like this:

resource "azurerm_resource_group" "main" {
  count    = length(data.azurerm_resources.exists.resources) == 0 ? 1 : 0
  name     = local.rg_name
  location = var.resource_group_location  

  lifecycle {
    ignore_changes = all
  }

  depends_on = [
    data.azurerm_resources.exists
  ]
}

resource "azurerm_storage_account" "main" {
  count = local.create ? 1 : 0

  name                = local.name_alphanum
  resource_group_name = azurerm_resource_group.main[0].name
  location            = azurerm_resource_group.main[0].location

  account_kind              = "BlobStorage"
  min_tls_version           = "TLS1_2"
  enable_https_traffic_only = true
  is_hns_enabled            = false
  nfsv3_enabled             = false
  shared_access_key_enabled = var.shared_access_key_enabled
  allow_blob_public_access  = false

  account_tier             = var.account_tier
  account_replication_type = var.account_replication_type

  lifecycle {
    ignore_changes = all
  }

  depends_on = [
    data.azurerm_resources.exists
  ]
}

resource "azurerm_storage_container" "main" {
  count = local.create ? 1 : 0

  name                  = local.container_name
  storage_account_name  = azurerm_storage_account.main[0].name
  container_access_type = "private"

  lifecycle {
    ignore_changes = all
  }

  depends_on = [
    data.azurerm_resources.exists
  ]
}

💪 pros

🥀 cons

Conclusion

This solution is so far from be the best or most appropriate for the problem. Because in addition to go against the principles of the standard terraform paradigm. It is impossible have a simple and standardized implementation.