search

aws-ia / terraform-aws-control_tower_account_factory

AWS Control Tower Account Factory
Apache License 2.0
604 stars 386 forks source link

New account not created but pipeline started when account_request.tf is committed #389

Closed monteroete closed 9 months ago

monteroete commented 9 months ago

Terraform Version & Prov: Terraform v1.5.7

AFT Version: 1.10.4

Terraform Version & Provider Versions Please provide the outputs of terraform version and terraform providers from within your AFT environment

terraform version 0.15.5

terraform providers

Bug Description

I have installed AFT yesterday with Terraform v1.5.7 from my laptop.

The pipeline manage to start when an account_request.tf is committed to the repository. But the pipeline ends after 1 minute 40 seconds. No account is created.

Line 181 of the log indicated that a file is not found.

Line 225 says that there is no change in the infrastructure. But new account should have been created.

Expected behavior A new account should be created

Related Logs

[Container] 2023/09/20 07:55:20 Waiting for agent ping -- 2 | [Container] 2023/09/20 07:55:21 Waiting for DOWNLOAD_SOURCE 3 | [Container] 2023/09/20 07:55:22 Phase is DOWNLOAD_SOURCE 4 | [Container] 2023/09/20 07:55:22 CODEBUILD_SRC_DIR=/codebuild/output/src3644181770/src 5 | [Container] 2023/09/20 07:55:22 YAML location is /codebuild/readonly/buildspec.yml 6 | [Container] 2023/09/20 07:55:22 Setting HTTP client timeout to higher timeout for S3 source 7 | [Container] 2023/09/20 07:55:22 Processing environment variables 8 | [Container] 2023/09/20 07:55:22 No runtime version selected in buildspec. 9 | [Container] 2023/09/20 07:55:22 Moving to directory /codebuild/output/src3644181770/src 10 | [Container] 2023/09/20 07:55:22 Configuring ssm agent with target id: codebuild:2dae7db4-acb5-4880-aede-0c27da51a45b 11 | [Container] 2023/09/20 07:55:22 Successfully updated ssm agent configuration 12 | [Container] 2023/09/20 07:55:22 Registering with agent 13 | [Container] 2023/09/20 07:55:22 Phases found in YAML: 3 14 | [Container] 2023/09/20 07:55:22 PRE_BUILD: 20 commands 15 | [Container] 2023/09/20 07:55:22 BUILD: 1 commands 16 | [Container] 2023/09/20 07:55:22 POST_BUILD: 1 commands 17 | [Container] 2023/09/20 07:55:22 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED 18 | [Container] 2023/09/20 07:55:22 Phase context status code: Message: 19 | [Container] 2023/09/20 07:55:22 Entering phase INSTALL 20 | [Container] 2023/09/20 07:55:22 Phase complete: INSTALL State: SUCCEEDED 21 | [Container] 2023/09/20 07:55:22 Phase context status code: Message: 22 | [Container] 2023/09/20 07:55:22 Entering phase PRE_BUILD 23 | [Container] 2023/09/20 07:55:22 Running command DEFAULT_PATH=$(pwd) 24 |   25 | [Container] 2023/09/20 07:55:22 Running command TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S') 26 |   27 | [Container] 2023/09/20 07:55:22 Running command AWS_MODULE_SOURCE=$(aws ssm get-parameter --name "/aft/config/aft-pipeline-code-source/repo-url" --query "Parameter.Value" --output text) 28 |   29 | [Container] 2023/09/20 07:55:37 Running command AWS_MODULE_GIT_REF=$(aws ssm get-parameter --name "/aft/config/aft-pipeline-code-source/repo-git-ref" --query "Parameter.Value" --output text) 30 |   31 | [Container] 2023/09/20 07:55:38 Running command TF_VERSION=$(aws ssm get-parameter --name "/aft/config/terraform/version" --query "Parameter.Value" --output text) 32 |   33 | [Container] 2023/09/20 07:55:38 Running command TF_DISTRIBUTION=$(aws ssm get-parameter --name "/aft/config/terraform/distribution" --query "Parameter.Value" --output text) 34 |   35 | [Container] 2023/09/20 07:55:39 Running command CT_MGMT_REGION=$(aws ssm get-parameter --name "/aft/config/ct-management-region" --query "Parameter.Value" --output text) 36 |   37 | [Container] 2023/09/20 07:55:39 Running command AFT_MGMT_ACCOUNT=$(aws sts get-caller-identity --query Account --output text) 38 |   39 | [Container] 2023/09/20 07:55:40 Running command AFT_EXEC_ROLE_ARN=arn:$AWS_PARTITION:iam::$AFT_MGMT_ACCOUNT:role/AWSAFTExecution 40 |   41 | [Container] 2023/09/20 07:55:40 Running command AFT_ADMIN_ROLE_NAME=$(aws ssm get-parameter --name /aft/resources/iam/aft-administrator-role-name \| jq --raw-output ".Parameter.Value") 42 |   43 | [Container] 2023/09/20 07:55:41 Running command AFT_ADMIN_ROLE_ARN=arn:$AWS_PARTITION:iam::$AFT_MGMT_ACCOUNT:role/$AFT_ADMIN_ROLE_NAME 44 |   45 | [Container] 2023/09/20 07:55:41 Running command ROLE_SESSION_NAME=$(aws ssm get-parameter --name /aft/resources/iam/aft-session-name \| jq --raw-output ".Parameter.Value") 46 |   47 | [Container] 2023/09/20 07:55:41 Running command ssh_key_parameter=$(aws ssm get-parameter --name /aft/config/aft-ssh-key --with-decryption 2> /dev/null \|\| echo "None") 48 | if [[ $ssh_key_parameter != "None" ]]; then 49 | ssh_key=$(jq --raw-output ".Parameter.Value" <<< $ssh_key_parameter) 50 | mkdir -p ~/.ssh 51 | echo "Host *" >> ~/.ssh/config 52 | echo "StrictHostKeyChecking no" >> ~/.ssh/config 53 | echo "UserKnownHostsFile=/dev/null" >> ~/.ssh/config 54 | echo "$ssh_key" > ~/.ssh/ssh_key 55 | echo -e "\n\n" >> ~/.ssh/ssh_key 56 | chmod 600 ~/.ssh/ssh_key 57 | eval "$(ssh-agent -s)" 58 | ssh-add ~/.ssh/ssh_key 59 | fi 60 |   61 |   62 | [Container] 2023/09/20 07:55:42 Running command git config --global credential.helper '!aws codecommit credential-helper $@' 63 |   64 | [Container] 2023/09/20 07:55:42 Running command git config --global credential.UseHttpPath true 65 |   66 | [Container] 2023/09/20 07:55:42 Running command git clone -b $AWS_MODULE_GIT_REF $AWS_MODULE_SOURCE aws-aft-core-framework 67 | Cloning into 'aws-aft-core-framework'... 68 | Note: switching to '6c0b356895478bb5c6578417640819aa0c8d774b'. 69 |   70 | You are in 'detached HEAD' state. You can look around, make experimental 71 | changes and commit them, and you can discard any commits you make in this 72 | state without impacting any branches by switching back to a branch. 73 |   74 | If you want to create a new branch to retain commits you create, you may 75 | do so (now or later) by using -c with the switch command. Example: 76 |   77 | git switch -c 78 |   79 | Or undo this operation with: 80 |   81 | git switch - 82 |   83 | Turn off this advice by setting config variable advice.detachedHead to false 84 |   85 |   86 | [Container] 2023/09/20 07:55:44 Running command python3 -m venv ./venv 87 |   88 | [Container] 2023/09/20 07:55:50 Running command source ./venv/bin/activate 89 |   90 | [Container] 2023/09/20 07:55:50 Running command pip install jinja2-cli==0.7.0 Jinja2==3.0.1 MarkupSafe==2.0.1 boto3==1.18.56 requests==2.26.0 91 | Collecting jinja2-cli==0.7.0 92 | Downloading jinja2_cli-0.7.0-py2.py3-none-any.whl (6.2 kB) 93 | Collecting Jinja2==3.0.1 94 | Downloading Jinja2-3.0.1-py3-none-any.whl (133 kB) 95 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.7/133.7 kB 26.5 MB/s eta 0:00:00 96 | Collecting MarkupSafe==2.0.1 97 | Downloading MarkupSafe-2.0.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (30 kB) 98 | Collecting boto3==1.18.56 99 | Downloading boto3-1.18.56-py3-none-any.whl (131 kB) 100 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 131.6/131.6 kB 2.6 MB/s eta 0:00:00 101 | Collecting requests==2.26.0 102 | Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB) 103 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.3/62.3 kB 2.2 MB/s eta 0:00:00 104 | Collecting jmespath<1.0.0,>=0.7.1 105 | Downloading jmespath-0.10.0-py2.py3-none-any.whl (24 kB) 106 | Collecting s3transfer<0.6.0,>=0.5.0 107 | Downloading s3transfer-0.5.2-py3-none-any.whl (79 kB) 108 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.5/79.5 kB 2.8 MB/s eta 0:00:00 109 | Collecting botocore<1.22.0,>=1.21.56 110 | Downloading botocore-1.21.65-py3-none-any.whl (8.0 MB) 111 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.0/8.0 MB 78.4 MB/s eta 0:00:00 112 | Collecting urllib3<1.27,>=1.21.1 113 | Downloading urllib3-1.26.16-py2.py3-none-any.whl (143 kB) 114 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 143.1/143.1 kB 4.1 MB/s eta 0:00:00 115 | Collecting certifi>=2017.4.17 116 | Downloading certifi-2023.7.22-py3-none-any.whl (158 kB) 117 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 kB 28.8 MB/s eta 0:00:00 118 | Collecting idna<4,>=2.5 119 | Downloading idna-3.4-py3-none-any.whl (61 kB) 120 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 1.1 MB/s eta 0:00:00 121 | Collecting charset-normalizer~=2.0.0 122 | Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 123 | Collecting python-dateutil<3.0.0,>=2.1 124 | Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB) 125 | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 42.4 MB/s eta 0:00:00 126 | Collecting six>=1.5 127 | Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 128 | Installing collected packages: urllib3, six, MarkupSafe, jmespath, idna, charset-normalizer, certifi, requests, python-dateutil, Jinja2, jinja2-cli, botocore, s3transfer, boto3 129 | Successfully installed Jinja2-3.0.1 MarkupSafe-2.0.1 boto3-1.18.56 botocore-1.21.65 certifi-2023.7.22 charset-normalizer-2.0.12 idna-3.4 jinja2-cli-0.7.0 jmespath-0.10.0 python-dateutil-2.8.2 requests-2.26.0 s3transfer-0.5.2 six-1.16.0 urllib3-1.26.16 130 |   131 | [notice] A new release of pip is available: 23.0.1 -> 23.2.1 132 | [notice] To update, run: pip install --upgrade pip 133 |   134 | [Container] 2023/09/20 07:55:55 Running command if [ $TF_DISTRIBUTION = "oss" ]; then 135 | TF_BACKEND_REGION=$(aws ssm get-parameter --name "/aft/config/oss-backend/primary-region" --query "Parameter.Value" --output text) 136 | TF_KMS_KEY_ID=$(aws ssm get-parameter --name "/aft/config/oss-backend/kms-key-id" --query "Parameter.Value" --output text) 137 | TF_DDB_TABLE=$(aws ssm get-parameter --name "/aft/config/oss-backend/table-id" --query "Parameter.Value" --output text) 138 | TF_S3_BUCKET=$(aws ssm get-parameter --name "/aft/config/oss-backend/bucket-id" --query "Parameter.Value" --output text) 139 | TF_S3_KEY=account-request/terraform.tfstate 140 | cd /tmp 141 | echo "Installing Terraform" 142 | curl -o terraform_${TF_VERSION}_linux_amd64.zip https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip 143 | unzip -o terraform_${TF_VERSION}_linux_amd64.zip && mv terraform /usr/bin 144 | terraform --version 145 | cd $DEFAULT_PATH/terraform 146 | for f in *.jinja; do jinja2 $f -D timestamp="$TIMESTAMP" -D tf_distribution_type=$TF_DISTRIBUTION -D provider_region=$CT_MGMT_REGION -D region=$TF_BACKEND_REGION -D bucket=$TF_S3_BUCKET -D key=$TF_S3_KEY -D dynamodb_table=$TF_DDB_TABLE -D kms_key_id=$TF_KMS_KEY_ID -D aft_admin_role_arn=$AFT_EXEC_ROLE_ARN >> ./$(basename $f .jinja).tf; done 147 | for f in *.tf; do echo "\n \n"; echo $f; cat $f; done 148 | JSON=$(aws sts assume-role --role-arn ${AFT_ADMIN_ROLE_ARN} --role-session-name ${ROLE_SESSION_NAME}) 149 | #Make newly assumed role default session 150 | export AWS_ACCESS_KEY_ID=$(echo ${JSON} \| jq --raw-output ".Credentials[\"AccessKeyId\"]") 151 | export AWS_SECRET_ACCESS_KEY=$(echo ${JSON} \| jq --raw-output ".Credentials[\"SecretAccessKey\"]") 152 | export AWS_SESSION_TOKEN=$(echo ${JSON} \| jq --raw-output ".Credentials[\"SessionToken\"]") 153 | terraform init -no-color 154 | else 155 | TF_ORG_NAME=$(aws ssm get-parameter --name "/aft/config/terraform/org-name" --query "Parameter.Value" --output text) 156 | TF_TOKEN=$(aws ssm get-parameter --name "/aft/config/terraform/token" --with-decryption --query "Parameter.Value" --output text) 157 | TF_ENDPOINT=$(aws ssm get-parameter --name "/aft/config/terraform/api-endpoint" --query "Parameter.Value" --output text) 158 | TF_WORKSPACE_NAME="ct-aft-account-request" 159 | TF_CONFIG_PATH="./temp_configuration_file.tar.gz" 160 | cd $DEFAULT_PATH/terraform 161 | for f in *.jinja; do jinja2 $f -D timestamp="$TIMESTAMP" -D provider_region=$CT_MGMT_REGION -D tf_distribution_type=$TF_DISTRIBUTION -D terraform_org_name=$TF_ORG_NAME -D terraform_workspace_name=$TF_WORKSPACE_NAME -D aft_admin_role_arn=$AFT_EXEC_ROLE_ARN >> ./$(basename $f .jinja).tf; done 162 | for f in *.tf; do echo "\n \n"; echo $f; cat $f; done 163 | cd $DEFAULT_PATH 164 | tar -czf temp_configuration_file.tar.gz -C terraform --exclude .git --exclude venv . 165 | python3 $DEFAULT_PATH/aws-aft-core-framework/sources/scripts/workspace_manager.py --operation "deploy" --organization_name $TF_ORG_NAME --workspace_name $TF_WORKSPACE_NAME --assume_role_arn $AFT_ADMIN_ROLE_ARN --assume_role_session_name $ROLE_SESSION_NAME --api_endpoint $TF_ENDPOINT --api_token $TF_TOKEN --terraform_version $TF_VERSION --config_file $TF_CONFIG_PATH 166 | fi 167 |   168 | Installing Terraform 169 | % Total % Received % Xferd Average Speed Time Time Time Current 170 | Dload Upload Total Spent Left Speed 171 |   172 | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 173 | 100 31.5M 100 31.5M 0 0 348M 0 --:--:-- --:--:-- --:--:-- 350M 174 | Archive: terraform_0.15.5_linux_amd64.zip 175 | inflating: terraform 176 | Terraform v0.15.5 177 | on linux_amd64 178 |   179 | Your version of Terraform is out of date! The latest version 180 | is 1.5.7. You can update by downloading from https://www.terraform.io/downloads.html 181 | /codebuild/output/tmp/script.sh: line 15: cd: /codebuild/output/src3644181770/src/terraform: No such file or directory 182 | Traceback (most recent call last): 183 | File "/codebuild/output/src3644181770/src/venv/bin/jinja2", line 8, in 184 | sys.exit(main()) 185 | File "/codebuild/output/src3644181770/src/venv/lib/python3.9/site-packages/jinja2cli/cli.py", line 424, in main 186 | sys.exit(cli(opts, args)) 187 | File "/codebuild/output/src3644181770/src/venv/lib/python3.9/site-packages/jinja2cli/cli.py", line 314, in cli 188 | out.write(render(template_path, data, extensions, opts.strict)) 189 | File "/codebuild/output/src3644181770/src/venv/lib/python3.9/site-packages/jinja2cli/cli.py", line 229, in render 190 | return env.get_template(os.path.basename(template_path)).render(data) 191 | File "/codebuild/output/src3644181770/src/venv/lib/python3.9/site-packages/jinja2/environment.py", line 997, in get_template 192 | return self._load_template(name, globals) 193 | File "/codebuild/output/src3644181770/src/venv/lib/python3.9/site-packages/jinja2/environment.py", line 958, in _load_template 194 | template = self.loader.load(self, name, self.make_globals(globals)) 195 | File "/codebuild/output/src3644181770/src/venv/lib/python3.9/site-packages/jinja2/loaders.py", line 125, in load 196 | source, filename, uptodate = self.get_source(environment, name) 197 | File "/codebuild/output/src3644181770/src/venv/lib/python3.9/site-packages/jinja2/loaders.py", line 214, in get_source 198 | raise TemplateNotFound(template) 199 | jinja2.exceptions.TemplateNotFound: *.jinja 200 | \n \n 201 | *.tf 202 |   203 | Initializing the backend... 204 |   205 | Initializing provider plugins... 206 |   207 | Terraform has been successfully initialized! 208 |   209 | You may now begin working with Terraform. Try running "terraform plan" to see 210 | any changes that are required for your infrastructure. All Terraform commands 211 | should now work. 212 |   213 | If you ever set or change modules or backend configuration for Terraform, 214 | rerun this command to reinitialize your working directory. If you forget, other 215 | commands will detect it and remind you to do so if necessary. 216 |   217 | [Container] 2023/09/20 07:56:00 Phase complete: PRE_BUILD State: SUCCEEDED 218 | [Container] 2023/09/20 07:56:00 Phase context status code: Message: 219 | [Container] 2023/09/20 07:56:00 Entering phase BUILD 220 | [Container] 2023/09/20 07:56:00 Running command if [ $TF_DISTRIBUTION = "oss" ]; then 221 | terraform apply -no-color --auto-approve 222 | fi 223 |   224 |   225 | No changes. Your infrastructure matches the configuration. 226 |   227 | Terraform has compared your real infrastructure against your configuration 228 | and found no differences, so no changes are needed. 229 |   230 | Apply complete! Resources: 0 added, 0 changed, 0 destroyed. 231 |   232 | [Container] 2023/09/20 07:56:00 Phase complete: BUILD State: SUCCEEDED 233 | [Container] 2023/09/20 07:56:00 Phase context status code: Message: 234 | [Container] 2023/09/20 07:56:00 Entering phase POST_BUILD 235 | [Container] 2023/09/20 07:56:00 Running command echo "Post-Build" 236 | Post-Build 237 |   238 | [Container] 2023/09/20 07:56:00 Phase complete: POST_BUILD State: SUCCEEDED 239 | [Container] 2023/09/20 07:56:00 Phase context status code: Message: 240 |  

Additional context Add any other context about the problem here.

adam-daily commented 9 months ago

Hi there. The best route in this case is to reach out to AWS Premium Support for assistance. They have more insight into your account than we do and will be better able to help you troubleshoot.