aws-ia / terraform-aws-control_tower_account_factory

AWS Control Tower Account Factory
Apache License 2.0
604 stars 386 forks source link

Issues running aft-customizations-execute-pipeline as part of aft-invoke-customizations step function #404

Closed rianjp closed 7 months ago

rianjp commented 8 months ago

Terraform Version & Prov: Terraform v1.2.6 AWS Provider v4.67.0 AFT Version: 1.10.4 (Can be found in the AFT Management Account in the SSM Parameter /aft/config/aft/version)

Terraform Version & Provider Versions Please provide the outputs of terraform version and terraform providers from within your AFT environment

terraform version

Terraform v1.2.6

terraform providers

AWS Provider v4.67.0

Bug Description aft-customizations-execute-pipeline produces a traceback when the aft-invoke-customizations step function is ran to invoke customizations on an account that hasn't has its customization pipeline ran for >1 year. This causes the step function to fail its execution.

To Reproduce Steps to reproduce the behavior:

  1. Execute the aft-invoke-customizations using the method described here using the following JSON, on an account where the last execution of the customization pipeline was >1 yr:
    {
    "include": [
      {
        "type": "accounts",
        "target_value": [
          "<acc_id>"
        ]
      }
    ]
    }
  2. The aft-customizations-execute-pipeline will produce a traceback to codepipeline.py in the aft_common lambda layer codepipeline.py pipeline_is_running function
  3. Verify in the aft-customizations-execute-pipeline lambdafunction logs that the error produced indicates "EXCEPTION": "list index out of range" for the lambda handler and the traceback points to line 58 of pipeline is running.

Expected behavior Handling of no results being returned for the startTime when assigning the last_execution variable for the current pipelineExecutionSummaries due to the pipeline not having ran in the last year. This expected behaviour has already been implemented in the same file here but appears to be missinng from the function causing the current error.

Related Logs JSON formatted log message from lambda function logs

{
    "time_stamp": "<timestamp>",
    "module": "aft_customizations_execute_pipeline",
    "log_level": "ERROR",
    "log_message": {
        "FILE": "aft_customizations_execute_pipeline.py",
        "METHOD": "lambda_handler",
        "EXCEPTION": "list index out of range"
    }
}

Traceback from lambda function logs

Traceback (most recent call last):
  File "/var/task/aft_customizations_execute_pipeline.py", line 38, in lambda_handler
    execute_pipeline(session, str(account_id))
  File "/opt/python/lib/python3.9/site-packages/aft_common/codepipeline.py", line 72, in execute_pipeline
    if not pipeline_is_running(session, name):
  File "/opt/python/lib/python3.9/site-packages/aft_common/codepipeline.py", line 58, in pipeline_is_running
    latest_execution = sorted(
IndexError: list index out of range

Additional context None

Sanjan611 commented 8 months ago

Hi @rianjp , thanks for bringing this to our notice! I've added an item in our backlog to address this bug.

stumins commented 7 months ago

Hi @rianjp,

We've released AFT 1.11.0 which includes a patch for this bug. Please update to the latest version of AFT & retry.

grs-schleupen commented 1 month ago

@stumins Is there a workaround to get the "expired" pipelines working without the update? Thx.