Update to version 3.0.0. failed

ErikMeinders commented 4 years ago

After committing the Pull Request into master, I got the following error during the build phase

2020-01-15 15:50:17,131 | INFO | main | ADF Version 3.0.0 | (main.py:235)

139 | 2020-01-15 15:50:17,131 | INFO | main | ADF Log Level is INFO | (main.py:236) 140 | 2020-01-15 15:50:18,167 | INFO | organization_policy | Determining if Organization Policy changes are required. (Tagging or Service Controls) | (organization_policy.py:97) 141 | 2020-01-15 15:50:36,115 | INFO | organizations | SERVICE_CONTROL_POLICY are currently enabled within the Organization | (organizations.py:49) 142 | Traceback (most recent call last): 143 | File "adf-build/main.py", line 348, in 144 | main() 145 | File "adf-build/main.py", line 251, in main 146 | policies.apply(organizations, parameter_store, config.config) 147 | File "/codebuild/output/src464162939/src/adf-build/organization_policy.py", line 115, in apply 148 | organizations 149 | File "/codebuild/output/src464162939/src/adf-build/organization_policy.py", line 63, in set_scp_attachment 150 | 'p-FullAWSAccess', organization_mapping[path]) 151 | KeyError: '' 152 | 153 | [Container] 2020/01/15 15:50:36 Command did not exit successfully python adf-build/main.py exit status 1 154 | [Container] 2020/01/15 15:50:36 Phase complete: BUILD State: FAILED 155 | [Container] 2020/01/15 15:50:36 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: python adf-build/main.py. Reason: exit status 1 156 | [Container] 2020/01/15 15:50:36 Entering phase POST_BUILD

bundyfx commented 4 years ago

Thanks @ErikMeinders - Looking into it now to see if I can reproduce it.

bundyfx commented 4 years ago

Hi @ErikMeinders - The error is because you still have an scp.json in the root of the bootstrap repository, with the 3.0 change the adf-bootstrap folder is now considered the root in regards to bootstrapping or applications of scps and base templates. Move the scp.json from the root of your repository into the adf-bootstrap folder.. I will make a hotfix asap to give an appropriate error message when there is an scp.json, template.yml or tagging-policy.json in the root rather than in the adf-bootstrap folder :).

Regards

ErikMeinders commented 4 years ago

Hi @bundyfx,

I moved the file, but the error remains. I moved the file into the adf-bootstrap as well as into the deployment directory in there when the first attempt (only in adf-bootstrap) failed.

Then, looking at the code and the scp value in the parameter store, I noticed that I had an extra SCP for one of my OUs. This needed to be moved to adf-bootstrap too.

The scp.json file now is in all 3 locations mentioned in the scp parameter. I assume these values are relative to adf-bootstrap!? ... ['./scp.json', './Data/scp.json', './deployment/scp.json']

As all scp.jsons are identical right now, I consider changing the value in the scp parameter ...

ErikMeinders commented 4 years ago

Last experiment for the day.

I cleaned the value of the scp parameter in parameter store into [], removed the attachments from the SCPs and deleted the SCPs themselves. With all scp.json files in place (see above) I re-ran the pipeline. It did complete with errors. Then I checked the SCPs and they were re-created and attached to the entities expected.

It does notice no changes need to be made and yet, apply() bails out at line 129...

What is the KeyError: '' referring to?

Running the pipeline again gave:

bundyfx commented 4 years ago

Hi @ErikMeinders

I believe I have found the issue.. Working on hotfix now :)

bundyfx commented 4 years ago

Hi @ErikMeinders

Please try 3.0.1 now, it is the latest version on the SAR. It should resolve this issue.

ErikMeinders commented 4 years ago

I ran into some other issues related to the fact that account had been moved into sub-OUs, but other than that, it works!

bundyfx commented 4 years ago

Thanks! Ill close this now but please open an issue for anything else you run into! :)

awslabs / aws-deployment-framework

Update to version 3.0.0. failed #210

2020-01-15 15:50:17,131 | INFO | main | ADF Version 3.0.0 | (main.py:235)