search

aztfmod / rover

The rover is a docker container in charge of the deployment of the Terraform platform engineering for Azure
MIT License
175 stars 147 forks source link

rover 'state rm': state file not found #292

Open gesnaud opened 2 years ago

gesnaud commented 2 years ago

Hello,

I'm facing an issue when trying to make a state rm in rover command:

rover -lz /tf/caf/.solution/caf/caf_solution \
-var-folder /tf/caf/my_var_folder \
-a 'state rm' \
-p ${PLAN_FILE_NAME} \
-level level4 \
-env ${LZ_PREFIX} \
-tfstate ${TFSTATE_FILE} \
-tfstate_subscription_id ${TFSTATE_SUB_ID} \
-parallelism 30 \
-target_subscription ${SUB_NAME} \
-id "module.solution.module.network_something[\"blabal\"].azurecaf_name.networksomething"`

And the result is a state file was found! error message::

@calling verify_azure_session
Checking existing Azure session
@calling process_target_subscription
Set subscription to -target_subscription SXXXXX01
caf_command landingzone
target_subscription_id 8XXXXXe
TF_VAR_tfstate_subscription_id 8XXXXXe
Resources from this landing zone are going to be deployed in the following subscription:
{
  "environmentName": "AzureCloud",
  "homeTenantId": "axxxxxx",
  "id": "xxxxxf",
  "isDefault": true,
  "managedByTenants": [],
  "name": "XXXXXX",
  "state": "Enabled",
  "tenantId": "xxxxxx",
  "user": {
    "name": "hey@you.onmicrosoft.com",
    "type": "user"
  }
}
debug: 6XXXXXXX
Tfstates subscription set to XXXX (XXXXXXXXXXX)

mode                          : 'landingzone'
terraform command output file : ''
terraform plan output file    : 'foobar_tfplan'
directory cache               : '/home/vscode/.terraform.cache/foobar_launchpad/rover_jobs/20221124112354655979305'
tf_action                     : 'state list'
command and parameters        : '-var-file=/tf/caf/environements/vxxxxx/landingzone.tfvars -var-file=/tf/caf/environements/xxxx/rgs.tfvars -var-file=/tf/caf/environements/xxxxx/resource.tfvars -parallelism 30'

level (current)               : 'level4'
environment                   : 'foobar_launchpad'
workspace                     : 'tfstate'
tfstate                       : 'foobar.tfstate'
tfstate subscription id       : 'XXXXX'
target subscription           : 'XXXXX'
CI/CD enabled                 : 'false'
Symphony Yaml file path       : ''
Run all tasks                 : 'true'
TF_IN_AUTOMATION              : 'true'

@calling process_actions
@calling verify_parameters
landingzone                   : '/tf/caf/landingzone'
@calling_deploy
@calling get_storage_id
@calling_get_logged_user_object_id
 - AZURE_ENVIRONMENT: AzureCloud
 - ARM_ENVIRONMENT: public
Initalizing az cloud variables
 - logged in user objectId: xxxxxx (hey@you.onmicrosoft.com)
Initializing state with user: hey@you.onmicrosoft.com

launchpad already installed

@calling deploy_from_remote_state
Connecting to the launchpad
@calling login_as_launchpad

Getting launchpad coordinates from subscription: 8xxxx9e
 - keyvault_name: vxxxxx-level4
 - tenant_id : axxxxxx8
 - storage_account_name (current): xxx4
 - storage_account_name (lower): xxx3
 - resource_group (current): xxxxlaunchpad-level4
 - resource_group (lower): xxxlaunchpad-level3
@calling deploy_landingzone
Deploying '/tf/caf/landingzone'
Terraform version 0.15 or greater
Upgrading modules...
Downloading aztfmod/caf/azurerm 5.4.8 for dynamic_keyvault_secrets...
- dynamic_keyvault_secrets in /home/vscode/.terraform.cache/foobar_launchpad/rover_jobs/20221112345679305/modules/dynamic_keyvault_secrets/modules/security/dynamic_keyvault_secrets
- dynamic_keyvault_secrets.secret in /home/vscode/.terraform.cache/foobar_launchpad/rover_jobs/20221112345679305/modules/dynamic_keyvault_secrets/modules/security/dynamic_keyvault_secrets/secret

Initializing the backend...

Successfully configured the backend "azurerm"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- terraform.io/builtin/terraform is built in to Terraform
- Finding hashicorp/azurerm versions matching "~> 2.81.0"...
- Finding latest version of hashicorp/local...
[...]
- Using hashicorp/tls v2.2.0 from the shared cache directory
- Using aztfmod/azurecaf v1.2.22 from the shared cache directory

Terraform has been successfully initialized!
Terraform init return code 0
calling other
@calling other
running terraform state rm -state=/home/vscode/.terraform.cache/foobar_launchpad/rover_jobs/20221112345679305/tfstates/levelX/tfstate/'foobar.tfstate module.solution.module.network_something[\"blabal\"].azurecaf_name.networksomething"
Terraform state rm return code: 0
Terraform returned errors:
No state file was found!

State management commands require a state file. Run this command
in a directory where Terraform has been run or use the -state flag
to point the command to a specific state location.
Error on or near line 555: Error running terraform state rm; exiting with status 2003

@calling clean_up_variables
cleanup variables
clean_up backend_files

But a state show is working fine:

rover -lz /tf/caf/.solution/caf/caf_solution \
-var-folder /tf/caf/my_var_folder \
-a 'state show' \
-p ${PLAN_FILE_NAME} \
-level level4 \
-env ${LZ_PREFIX} \
-tfstate ${TFSTATE_FILE} \
-tfstate_subscription_id ${TFSTATE_SUB_ID} \
-parallelism 30 \
-target_subscription ${SUB_NAME} \
-id "module.solution.module.network_something[\"blabal\"].azurecaf_name.networksomething"`

And the result is:

Terraform has been successfully initialized!
Terraform init return code 0
calling other
@calling other
running terraform state list -state=/home/vscode/.terraform.cache/foobar_launchapd/rover_jobs/2022114158234788/tfstates/level4/tfstate/foobar.tfstate module.solution.module.network_something["blabal"].azurecaf_name.networksomething

module.solution.module.network_something[\"blabal\"].azurecaf_name.networksomething"`
Terraform state list return code: 0

@calling clean_up_variables
cleanup variables
clean_up backend_files

real    0m39.992s
user    0m20.880s
sys     0m8.836s

My questions are:

My environment

gesnaud commented 2 years ago

Hi there o/

I forget to precise that i've made little enhancement on rover's scripts to be able to manage the id resource:

- rover/rover.sh

[...]
        -a|--action)
            export tf_action=$(parameter_value --action "${2}")
            shift 2
            ;;
        -id)
            export tf_id="${2}"
            shift 2
[...]

- rover/tfstate_azurerm.sh

function other {
    echo "@calling other"
    echo "running terraform ${tf_action} -state="${TF_DATA_DIR}/tfstates/${TF_VAR_level}/${TF_VAR_workspace}/${TF_VAR_tf_name}" ${tf_id}"
    download_tfstate
    rm -f $STDERR_FILE

    terraform \
        ${tf_action} \
        -state="${TF_DATA_DIR}/tfstates/${TF_VAR_level}/${TF_VAR_workspace}/${TF_VAR_tf_name}" \
        ${tf_id} 2>$STDERR_FILE | tee ${tf_output_file}
        #${tf_command} 2>$STDERR_FILE | tee ${tf_output_file}

    RETURN_CODE=${PIPESTATUS[0]} && echo "Terraform ${tf_action} return code: ${RETURN_CODE}"

 #   upload_tfstate
[...]

You could notice that I add upload_tfstate and download_tfstate to workaround the no state file. I comment it for now to avoid effects for other commands.

Thanks!

gesnaud commented 2 years ago

Here is a proposal for a clean 'patch' for tfstate_azurerm.sh. You will notice that I have to use a 'swap' file ( /tmp/tf_pipe_code)as I cannot get variable value from a subshell to the parent 🤔.

rover/rover.sh

[...]
        -a|--action)
            export tf_action=$(parameter_value --action "${2}")
            shift 2
            ;;
        -id)
            export tf_id="${2}"
            shift 2
[...]

tfstate_azurerm.sh

function other {
    echo "@calling other"

    rm -f $STDERR_FILE

    [[ "${tf_action}" == *"rm"* ]] && \
      (echo "tf_action *SHOULD* containing rm: "${tf_action};
          download_tfstate)

    [[ "${tf_action}" == *"state"* ]] && \
      (echo "running terraform ${tf_action} -state="${TF_DATA_DIR}/tfstates/${TF_VAR_level}/${TF_VAR_workspace}/${TF_VAR_tf_name}"  ${tf_id}"; 
       terraform \
        ${tf_action} \
        -state="${TF_DATA_DIR}/tfstates/${TF_VAR_level}/${TF_VAR_workspace}/${TF_VAR_tf_name}" \
        ${tf_id} \
        2>$STDERR_FILE | tee ${tf_output_file};
        echo ${PIPESTATUS[0]} > /tmp/tf_pipe_code);

    [[ "${tf_action}" != *"state"* ]] && \
      (echo "running terraform ${tf_action} -state="${TF_DATA_DIR}/tfstates/${TF_VAR_level}/${TF_VAR_workspace}/${TF_VAR_tf_name}" ${tf_command}"; 
       terraform \
        ${tf_action} \
        -state="${TF_DATA_DIR}/tfstates/${TF_VAR_level}/${TF_VAR_workspace}/${TF_VAR_tf_name}" \
        ${tf_command} \
        2>$STDERR_FILE | tee ${tf_output_file};
        echo ${PIPESTATUS[0]} > /tmp/tf_pipe_code);

    TF_PIPE_CODE=$(cat /tmp/tf_pipe_code);
    rm -f /tmp/tf_pipe_code;
    RETURN_CODE=${TF_PIPE_CODE} && echo "Terraform ${tf_action} return code: ${RETURN_CODE}"

    if [ -s $STDERR_FILE ]; then
        if [ ${tf_output_file+x} ]; then cat $STDERR_FILE >>${tf_output_file}; fi
        echo "Terraform returned errors:"
        cat $STDERR_FILE
        RETURN_CODE=2003
    fi

    if [ $RETURN_CODE != 0 ]; then
        error ${LINENO} "Error running terraform ${tf_action}" $RETURN_CODE
    else
        [[ "${tf_action}" == *"rm"* ]] && upload_tfstate
    fi
    echo "" 
}

@LaurentLesle I was thinking to make a pull request on the branch concerning aztfmod/rover:1.0.7-2109.2410 but I cannot find the matching info? Where is the matrix version of github aztfmod/rover giving dockerhub aztfmod/rover?

gesnaud commented 1 year ago

Hi Jason!

In your rover command line, you specified the resources address right?

Thx, Greg

----- Mail original ----- De: "Jason Dossett" @.> À: "aztfmod/rover" @.> Cc: "Gregory ESNAUD" @.>, "Mention" @.> Envoyé: Jeudi 26 Janvier 2023 16:53:42 Objet: Re: [aztfmod/rover] rover 'state rm': state file not found (Issue #292)

@gesnaud When I try your changes above, I get "At least one address is required." It doesn't look like the address is being included in the command call: running terraform state rm -state=/home/vscode/.terraform.cache/sepazcafenv/rover_jobs/20230126154929728220187/tfstates/level3/tfstate/lz_common_subscriptions.tfstate Usage: terraform [global options] state rm [options] ADDRESS...

Did I miss something?

— Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you were mentioned. Message ID: <aztfmod/rover/issues/292/1405218660 @ github . com>

jtdossett commented 1 year ago

I did not but I quickly figured out that’s what I was doing wrong which is why I deleted the question. :)

Thanks for the patch, it was very helpful!

On Jan 27, 2023, at 4:48 AM, Gregory ESNAUD @.***> wrote:



Hi Jason!

In your rover command line, you specified the resources address right?

Thx, Greg

----- Mail original ----- De: "Jason Dossett" @.> À: "aztfmod/rover" @.> Cc: "Gregory ESNAUD" @.>, "Mention" @.> Envoyé: Jeudi 26 Janvier 2023 16:53:42 Objet: Re: [aztfmod/rover] rover 'state rm': state file not found (Issue #292)

@gesnaud When I try your changes above, I get "At least one address is required." It doesn't look like the address is being included in the command call: running terraform state rm -state=/home/vscode/.terraform.cache/sepazcafenv/rover_jobs/20230126154929728220187/tfstates/level3/tfstate/lz_common_subscriptions.tfstate Usage: terraform [global options] state rm [options] ADDRESS...

Did I miss something?

— Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you were mentioned. Message ID: <aztfmod/rover/issues/292/1405218660 @ github . com>

— Reply to this email directly, view it on GitHubhttps://github.com/aztfmod/rover/issues/292#issuecomment-1406264380, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AU2J37YHQC6AWCNXK76NQFTWUOKVTANCNFSM6AAAAAASKN36BE. You are receiving this because you commented.Message ID: @.***>

gesnaud commented 1 year ago

Great :)

Have a nice day!

----- Mail original ----- De: "Jason Dossett" @.> À: "aztfmod/rover" @.> Cc: "Gregory ESNAUD" @.>, "Mention" @.> Envoyé: Vendredi 27 Janvier 2023 11:51:54 Objet: Re: [aztfmod/rover] rover 'state rm': state file not found (Issue #292)

I did not but I quickly figured out that’s what I was doing wrong which is why I deleted the question. :)

Thanks for the patch, it was very helpful!

On Jan 27, 2023, at 4:48 AM, Gregory ESNAUD @.***> wrote:

Hi Jason!

In your rover command line, you specified the resources address right?

Thx, Greg

----- Mail original ----- De: "Jason Dossett" @.> À: "aztfmod/rover" @.> Cc: "Gregory ESNAUD" @.>, "Mention" @.> Envoyé: Jeudi 26 Janvier 2023 16:53:42 Objet: Re: [aztfmod/rover] rover 'state rm': state file not found (Issue #292)

@gesnaud When I try your changes above, I get "At least one address is required." It doesn't look like the address is being included in the command call: running terraform state rm -state=/home/vscode/.terraform.cache/sepazcafenv/rover_jobs/20230126154929728220187/tfstates/level3/tfstate/lz_common_subscriptions.tfstate Usage: terraform [global options] state rm [options] ADDRESS...

Did I miss something?

— Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you were mentioned. Message ID: <aztfmod/rover/issues/292/1405218660 @ github . com>

— Reply to this email directly, view it on GitHubhttps://github.com/aztfmod/rover/issues/292#issuecomment-1406264380, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AU2J37YHQC6AWCNXK76NQFTWUOKVTANCNFSM6AAAAAASKN36BE. You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you were mentioned. Message ID: <aztfmod/rover/issues/292/1406332714 @ github . com>