An error occurred while executing the state 'run_create_pipeline?' (entered at the event id #33). Invalid path '$.Input.account_provisioning.run_create_pipeline': The choice state's condition path references an invalid value."

[vneekhra]

Terraform Version & Prov:

AFT Version: 1.9.1

Terraform Version & Provider Versions Please provide the outputs of terraform version and terraform providers from within your AFT environment

terraform version

0.15.5

terraform providers

AWS

Bug Description We have deployed control tower and AFT for terraform in a separate AFT account using Terraform, aft version 1.9.1. After deploying aft new account request is working fine, it is running pipeline for creating the account whenever we add new account request terraform code in our AFT account request repository. But account customisation is not working and even we can't see the state machine for account-provisioning-customization as well as no pipeline for any of the account created for account customisation. When we try to run the aft-invoke-customization step function then we are getting below error. Note: The logs mentions about account creation but the account is already existing and we are making customisation through account-customization.

{ "Cause": "An error occurred while executing the state 'run_create_pipeline?' (entered at the event id #33). Invalid path '$.Input.account_provisioning.run_create_pipeline': The choice state's condition path references an invalid value.", "Error": "States.Runtime", "ExecutionArn": "arn:aws:states:us-east-2::execution:aft-account-provisioning-framework:e5c48973-f6fa-4def-beaf-55ca11e33ba2", "Input": "{\"account_info\":{\"account\":{\"id\":\"\",\"email\":\"shared_acct@email\",\"name\":\"shared-account\", \"joined_method\":\"CREATED\",\"joined_date\":\"2023-03-09 07:51:44.747000+00:00\",\"status\":\"ACTIVE\",\"parent_id\":\"ou-38lh-9att8jja\",\"parent_type\":\"ORGANIZATIONAL_UNIT\", \"type\":\"account\",\"vendor\":\"aws\"}},\"control_tower_event\":{},\"account_request\":{\"custom_fields\":\"{\\"group\\":\\"prod\\"}\",\"change_management_parameters\": {\"change_reason\":\"Create new ControlPlane account shared-account\",\"change_requested_by\":\"shared_acct@email.com\"},\"id\":\"shared_acct@email.com\",\"control_tower_parameters\": {\"AccountEmail\":\"sharedservices-account@email\",\"SSOUserFirstName\":\"-sharedservices-account\",\"SSOUserLastName\":\"sharedservices-account\" ,\"ManagedOrganizationalUnit\":\"controlplane-ou\",\"AccountName\":\"shared-account\",\"SSOUserEmail\":\"shared_acct@email.com@email\"},\"account_tags\": {\"Environment\":\"prod\",\"Owner\":\"sharedservices-account sharedservices-account\",\"Project\":\"xyz\",\"Vended\":\"true\",\"created_by\":\" sharedservices-account@email\"},\"account_customizations_name\":\"shared-customizations\"},\"account_provisioning\":{\"run_create_pipeline\":\"true\"}, \"customization_request_id\":\"c0bb8f9a-9f82-4c30-a62c-96119a391b53\"}", "InputDetails": { "Included": true }, "Name": "e5c48973-f6fa-4def-beaf-55ca11e33ba2", "StartDate": 1679307003825, "StateMachineArn": "arn:aws:states:us-east-2::stateMachine:aft-account-provisioning-framework", "Status": "FAILED", "StopDate": 1679307036829 }

To Reproduce Steps to reproduce the behavior:

1. Add terraform code in account-customization repository under account_customization_name valued folder
2. Run the Step function with below input { "include": [ { "type": "accounts", "target_value": [ "" ] } ] }

Expected behavior It should deploy resources mentioned in the Terraform in the target account.

Related Logs Please let us know if the AFT deployment logs are required or any other logs from the cloud watch I can provide.

Additional context Add any other context about the problem here.

[snebhu3]

@vneekhra , a missing state machine could mean that the AFT deployment was not successful. Could you confirm that AFT was deployed successfully and no errors were encountered?

[ahmedfourti]

got the same issue, my AFT deployment was successful without any error.

[rupadhy3]

@snebhu3 , The AFT deploymnet was successful and it has deployed around 325 resources ... no errors were reported in aft deployment. And for depoying AFT we have used the latest terraform version 1.4.2 (so it is indeed higher then 0.15.x). And even now if I try redeploying aft it reports no changes .... below is the snippet of the aft deployment:

module.aft.module.aft_customizations.aws_iam_role_policy.aft_invoke_customizations_sfn: Refreshing state... [id=aft-invoke-customizations-execution-role:aft-invoke-customizations-policy]
module.aft.module.aft_customizations.aws_iam_role_policy.aft_identify_targets_lambda: Refreshing state... [id=aft-identify-targets-execution-role:aft-identify-targets-policy]
module.aft.module.aft_customizations.aws_sfn_state_machine.aft_invoke_customizations_sfn: Refreshing state... [id=arn:aws:states:us-east-2:<12-digit-aft-account-id>:stateMachine:aft-invoke-customizations]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.
Releasing state lock. This may take a few moments...

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

aft_management_account_id = "<12-digit-aft-account-id>"
audit_account_id = "<12-digit-audit-account-id>"
ct_management_account_id = "<12-digit-ct-account-id>"
log_archive_account_id = "<12-digit-logarchive-account-id>"
region = "us-east-2"
+ [[ -f terraform.tfstate ]]_
```_

[ahmedfourti]

@rupadhy3 I got it working by adding following files here in the repo aft-account-provisioning-customizations which will create the missing state.

[rupadhy3]

@ahmedfourti Thanks for info, the issue is resolved. Though these files were already there in our aft-account-provisioning-customizations repository, but somereason they were not picked up during AFT deployment. So we just had to make a dummy commit into the repository and that triggered aft to deploy the state machine from our aft-account-provisioning-customizations.

Though the current issue is resolved but the I still see a bug ... that AFT during deployment should take the repository content and deploy it, it should not explicitly wait for another commit into the repository to deploy the state machine, if the files are already present.

Regards

[snebhu3]

@rupadhy3 thank you for your response. This should not be the case ideally. For understanding the root cause, a deeper troubleshooting will be required in the CT environment. Please reach out to AWS Premium Support for assistance on understanding the root cause.

Closing the issue since the issue has been resolved.

[glyhood]

For me, after I created/updated the account-provisioning-customizations repository in GitHub, the final missing step was to update my codestar-connections app to include this new repository.