gruntwork-io / terragrunt-action

A GitHub Action for installing and running Terragrunt
Apache License 2.0
99 stars 38 forks source link

will not initialize #75

Closed williamohara closed 1 month ago

williamohara commented 1 month ago

'''Describe the bug unable to initialize terraform from within Github action. I am new to both gh actions and to terragrunt. When i run any terragrunt command on my local it works fine but when i run on github actons with the configurations below i get this error when it reaches the Init step - I tried with any sub command and the same issue.

with:
    tf_version: 1.9.4
    tg_version: 0.66.[3](https://github.com/subscripify/base-infra/actions/runs/10335050372/job/28609166926#step:4:3)
    tg_dir: /github/workspace/live/prod-eastus/core-infra
    tg_command: init -migrate-state
    tg_comment: 0
    tg_add_approve: 1
  env:
    tf_version: 1.9.[4](https://github.com/subscripify/base-infra/actions/runs/10335050372/job/28609166926#step:4:4)
    tg_version: 0.66.3
    working_dir: /github/workspace
    TF_VAR_SUBSCRIPIFY_ADMIN_GROUP_AZURE_ENTRA_OBJECT_ID: ***
    TF_VAR_SUBSCRIPIFY_DB_FIRST_TENANT_PW: ***
    TF_VAR_SUBSCRIPIFY_KRATOS_DEFAULT_SECRET: ***
    TF_VAR_SUBSCRIPIFY_STORAGE_CONTAINER_NAME: ***
    TF_VAR_SUBSCRIPIFY_STORAGE_ACCOUNT_RG: ***
    TF_VAR_SUBSCRIPIFY_DB_ADMIN_UN: ***
    TF_VAR_SUBSCRIPIFY_DEV_SUB_ID: ***
    TF_VAR_SUBSCRIPIFY_DB_ADMIN_PW: ***
    TF_VAR_SUBSCRIPIFY_KRATOS_FIRST_TENANT_COURIER: ***
    TF_VAR_SUBSCRIPIFY_KRATOS_COOKIE_SECRET: ***
    TF_VAR_SUBSCRIPIFY_STORAGE_ACCOUNT_NAME: ***
/usr/bin/docker run --name e[5](https://github.com/subscripify/base-infra/actions/runs/10335050372/job/28609166926#step:4:5)da15357e105fb24eafbe893234e4c088dc_1b35d3 --label 25e5da --workdir /github/workspace --rm -e "tf_version" -e "tg_version" -e "working_dir" -e "TF_VAR_SUBSCRIPIFY_ADMIN_GROUP_AZURE_ENTRA_OBJECT_ID" -e "TF_VAR_SUBSCRIPIFY_DB_FIRST_TENANT_PW" -e "TF_VAR_SUBSCRIPIFY_KRATOS_DEFAULT_SECRET" -e "TF_VAR_SUBSCRIPIFY_STORAGE_CONTAINER_NAME" -e "TF_VAR_SUBSCRIPIFY_STORAGE_ACCOUNT_RG" -e "TF_VAR_SUBSCRIPIFY_DB_ADMIN_UN" -e "TF_VAR_SUBSCRIPIFY_DEV_SUB_ID" -e "TF_VAR_SUBSCRIPIFY_DB_ADMIN_PW" -e "TF_VAR_SUBSCRIPIFY_KRATOS_FIRST_TENANT_COURIER" -e "TF_VAR_SUBSCRIPIFY_KRATOS_COOKIE_SECRET" -e "TF_VAR_SUBSCRIPIFY_STORAGE_ACCOUNT_NAME" -e "INPUT_TF_VERSION" -e "INPUT_TG_VERSION" -e "INPUT_TG_DIR" -e "INPUT_TG_COMMAND" -e "INPUT_TOFU_VERSION" -e "INPUT_TG_COMMENT" -e "INPUT_TG_ADD_APPROVE" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_REPOSITORY_OWNER_ID" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_REPOSITORY_ID" -e "GITHUB_ACTOR_ID" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKFLOW_REF" -e "GITHUB_WORKFLOW_SHA" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_ENVIRONMENT" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e "ACTIONS_RESULTS_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/base-infra/base-infra":"/github/workspace" 25e5da:15357e105fb24eafbe893234e4c088dc
2024-08-10 22:58:42 Starting Terragrunt Action
2024-08-10 22:58:42 Installing Terraform version 1.9.4
cloning https://github.com/asdf-community/asdf-hashicorp.git
mise plugin:terraform ✓ https://github.com/asdf-community/asdf-hashicorp.git#f1[6](https://github.com/subscripify/base-infra/actions/runs/10335050372/job/28609166926#step:4:6)02b6
installing
~/.local/share/mise/plugins/terraform/bin/install
Downloading terraform version 1.9.4 from https://releases.hashicorp.com/terraform/1.9.4/terraform_1.9.4_linux_amd64.zip
Skipping verifying signatures and checksums either because gpg is not installed or explicitly skipped with ASDF_HASHICORP_SKIP_VERIFY
Cleaning terraform previous binaries
Creating terraform bin directory
Extracting terraform archive
mise terraform@1.9.4 ✓ installed
mise ~/.config/mise/config.toml tools: terraform@1.9.4
2024-08-10 22:58:43 Installing Terragrunt version 0.66.3
cloning https://github.com/ohmer/asdf-terragrunt.git
mise plugin:terragrunt ✓ https://github.com/ohmer/asdf-terragrunt.git#4a6651a
installing
~/.local/share/mise/plugins/terragrunt/bin/install
Downloading terragrunt from https://github.com/gruntwork-io/terragrunt/releases/download/v0.66.3/terragrunt_linux_amd64
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100 [7](https://github.com/subscripify/base-infra/actions/runs/10335050372/job/28609166926#step:4:7)9.0M  100 79.0M    0     0   105M      0 --:--:-- --:--:-- --:--:--  105M
mise terragrunt@0.66.3 ✓ installed
mise ~/.config/mise/config.toml tools: terragrunt@0.66.3
╷
│ Error: Backend initialization required, please run "terraform init"
│ 
│ Reason: Initial configuration of the requested backend "azurerm"
│ 
│ The "backend" is the interface that Terraform uses to store state,
│ perform operations, etc. If this message is showing up, it means that the
│ Terraform configuration you're using is using a custom configuration for
│ the Terraform backend.
│ 
│ Changes to backend configurations require reinitialization. This allows
│ Terraform to set up the new configuration, copy existing state, etc. Please
│ run
│ "terraform init" with either the "-reconfigure" or "-migrate-state" flags
│ to
│ use the current configuration.
│ 
│ If the change reason above is incorrect, please verify your configuration
│ hasn't changed and try again. At this point, no changes to your existing
│ configuration or state have been made.
╵
time=2024-0[8](https://github.com/subscripify/base-infra/actions/runs/10335050372/job/28609166926#step:4:8)-10T22:58:44Z level=error msg=[/github/workspace/live/prod-eastus/base-infra/.terragrunt-cache/2544460077] exit status 1

To Reproduce Steps to reproduce the behavior, code snippets and examples which can be used to reproduce the issue.

This is my directory structure (i took extras out)

.
├── README.md
├── live
│   ├── prod-eastus
│   │   ├── base-infra
│   │   │   └── terragrunt.hcl
│   │   ├── core-infra
│   │       └── terragrunt.hcl
│   └── terragrunt.hcl
└── modules
     └── core-infra
         └── main.tf

terragrunt.hcl in /live directoiry

remote_state {
  backend = "azurerm"
  config = {
    key                  = "${path_relative_to_include()}/terraform.infra.tfstate"
    resource_group_name  = "${get_env("TF_VAR_SUBSCRIPIFY_STORAGE_ACCOUNT_RG")}"
    storage_account_name = "${get_env("TF_VAR_SUBSCRIPIFY_STORAGE_ACCOUNT_NAME")}"
    container_name       = "tfstate"
  }
}

terragrunt.hcl in /core-infra

include {
  path = find_in_parent_folders()
}

terraform {
  source = "../../../modules/core-infra"
}

dependency base_infra {
  config_path = "../base-infra"
}

inputs = {

  base_infra_resource_group_name = dependency.base_infra.outputs.base_infra_resource_group_name

  k8_cluster_name                    = "core-cluster"
  db_server_name                     = "core-subscripify-db-server"
  repo_tag                           = "core-infra"
  core_vault_name                    = dependency.base_infra.outputs.core_vault_name
  core_rg_vnet_name                  = dependency.base_infra.outputs.core_rg_vnet_name
  azure_k8_subnet_name               = dependency.base_infra.outputs.azure_k8_subnet_name
  azure_private_resolver_subnet_name = dependency.base_infra.outputs.azure_private_resolver_subnet_name
  azure_resource_subnet_name         = dependency.base_infra.outputs.azure_services_subnet_name
  db_un_secret_name                  = dependency.base_infra.outputs.db_un_secret_name
  db_pw_secret_name                  = dependency.base_infra.outputs.db_pw_secret_name
  subscripify_cont_reg_name          = dependency.base_infra.outputs.subscripify_cont_reg_name
  location                           = "eastus"
  azure_sub_id                       = "${get_env("TF_VAR_SUBSCRIPIFY_DEV_SUB_ID")}"
  iac_version                        = "v0.3.0"
  base_infra_resource_group_name     = "base-infra-rg"
  core_infra_resource_group_name     = "core-infra-rg"

}

this is my gh action yaml:

name: Infra Deploy

on:
  push:
    branches:
      - main

env:
  tf_version: '1.9.4'
  tg_version: '0.66.3'
  working_dir: '/github/workspace'
  TF_VAR_SUBSCRIPIFY_ADMIN_GROUP_AZURE_ENTRA_OBJECT_ID: ${{ secrets.TF_VAR_SUBSCRIPIFY_ADMIN_GROUP_AZURE_ENTRA_OBJECT_ID }}
  TF_VAR_SUBSCRIPIFY_DB_FIRST_TENANT_PW: ${{ secrets.TF_VAR_SUBSCRIPIFY_DB_FIRST_TENANT_PW }}
  TF_VAR_SUBSCRIPIFY_KRATOS_DEFAULT_SECRET: ${{ secrets.TF_VAR_SUBSCRIPIFY_KRATOS_DEFAULT_SECRET }}
  TF_VAR_SUBSCRIPIFY_STORAGE_CONTAINER_NAME: ${{ secrets.TF_VAR_SUBSCRIPIFY_STORAGE_CONTAINER_NAME }}
  TF_VAR_SUBSCRIPIFY_STORAGE_ACCOUNT_RG: ${{ secrets.TF_VAR_SUBSCRIPIFY_STORAGE_ACCOUNT_RG }}
  TF_VAR_SUBSCRIPIFY_DB_ADMIN_UN: ${{ secrets.TF_VAR_SUBSCRIPIFY_DB_ADMIN_UN }}
  TF_VAR_SUBSCRIPIFY_DEV_SUB_ID: ${{ secrets.TF_VAR_SUBSCRIPIFY_DEV_SUB_ID }}
  TF_VAR_SUBSCRIPIFY_DB_ADMIN_PW: ${{ secrets.TF_VAR_SUBSCRIPIFY_DB_ADMIN_PW }}
  TF_VAR_SUBSCRIPIFY_KRATOS_FIRST_TENANT_COURIER: ${{ secrets.TF_VAR_SUBSCRIPIFY_KRATOS_FIRST_TENANT_COURIER }}
  TF_VAR_SUBSCRIPIFY_KRATOS_COOKIE_SECRET: ${{ secrets.TF_VAR_SUBSCRIPIFY_KRATOS_COOKIE_SECRET }}
  TF_VAR_SUBSCRIPIFY_STORAGE_ACCOUNT_NAME: ${{ secrets.TF_VAR_SUBSCRIPIFY_STORAGE_ACCOUNT_NAME }}

jobs:
  setup-env-and-required-items:
    runs-on: ubuntu-latest

    steps:  
    - name: Checkout repository
      uses: actions/checkout@v2

    - name: Install Azure CLI
      uses: azure/CLI@v1
      with:
        inlineScript: |
          # Add your inline script here

    - name: Azure CLI Login
      run: az login --service-principal -u ${{ secrets.AZURE_SERVICE_ACCOUNT_4_GH_ACTIONS_ID }} -p ${{ secrets.AZURE_SERVICE_ACCOUNT_4_GH_ACTIONS_PW }} --tenant ${{ secrets.AZURE_TENANT_ID }}

    - name: Install Helm
      run: |
        curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

  checks:
    runs-on: ubuntu-latest
    needs: [ setup-env-and-required-items ]
    steps:
      - name: 'Checkout'
        uses: actions/checkout@main

      - name: Check terragrunt HCL
        uses: gruntwork-io/terragrunt-action@v2
        with:
          tf_version: ${{ env.tf_version }}
          tg_version: ${{ env.tg_version }}
          tg_dir: ${{ env.working_dir }}/live/prod-eastus/core-infra
          tg_command: 'hclfmt --terragrunt-check --terragrunt-diff'

  init:
    runs-on: ubuntu-latest
    needs: [ setup-env-and-required-items, checks ]

    steps:
      - name: 'Checkout'
        uses: actions/checkout@main

      - name: Init
        uses: gruntwork-io/terragrunt-action@v2
        with:
          tf_version: ${{ env.tf_version }}
          tg_version: ${{ env.tg_version }}
          tg_dir: ${{ env.working_dir }}/live/prod-eastus/core-infra
          tg_command: 'init -migrate-state'

Expected behavior A clear and concise description of what you expected to happen.

Nice to have

Versions

Additional context Add any other context about the problem here.

williamohara commented 1 month ago

I think i found what i was doing wrong -it looks like I had mistakenly relied on the AZ cli within my worker - this had noting to do with terragrunt