ERROR: Task OrganizationUpdate execute failed. reason: Cannot read property 'serviceControlPolicies' of undefined

kaihendry commented 2 years ago

Subject of the issue

Unable to apply organizational changes

Your environment

0.9.19
AWS Code pipeline

Steps to reproduce

https://s.natalian.org/2022-05-25/organization.yml

Expected behaviour

No error

Actual behaviour

[Container] 2022/05/25 04:34:10 Running command org-formation perform-tasks ./organization-tasks.yml --no-color --state-bucket-name organization-formation-705671790868 --state-object state.json
--
41 | INFO: Executing: include 000-organization-build/organization-tasks.yml.
42 | INFO: Executing: update-organization organization.yml.
43 | ERROR: Task OrganizationUpdate execute failed. reason: Cannot read property 'serviceControlPolicies' of undefined
44 | Cannot read property 'serviceControlPolicies' of undefined (use option --print-stack to print stack)
45 | ERROR:
46 | ERROR: ==========================
47 | ERROR: Stopped performing task(s)
48 | ERROR: Following tasks failed:
49 | ERROR:  - Task OrganizationUpdate
50 | ERROR: Following tasks were not executed:
51 | ERROR:  - Task OrganizationBuildPipeline
52 | ERROR: ==========================
53 | ERROR:
54 | ERROR: Task OrganizationBuild execute failed. reason: Number of failed tasks 1 exceeded tolerance for failed tasks 0.
55 | ERROR:
56 | ERROR: ==========================
57 | ERROR: Stopped performing task(s)
58 | ERROR: Following tasks failed:
59 | ERROR:  - Task OrganizationBuild
60 | ERROR: Following tasks were not executed:
61 | ERROR:  - Task Types
62 | ERROR:  - Task AWSSSO
63 | ERROR: ==========================
64 | ERROR:
65 | ERROR: Number of failed tasks 1 exceeded tolerance for failed tasks 0.

OlafConijn commented 2 years ago

did you modify your state.json file by any change? org.yml seems as plain as can be. otherwise, use option --print-stack to print stack? that would be helpful too. thanks!

kaihendry commented 2 years ago

(ins)hendry-tw-mbp~/sorg/organization-formation$ npx org-formation print-tasks ./organization-tasks.yml --output yaml --max-concurrent-stacks 100 --max-concurrent-tasks 100
INFO: Executing: update-organization organization.yml.
WARN: AccessDenied: unable to log into account 914678715711. This might have various causes, to troubleshoot:
https://github.com/OlafConijn/AwsOrganizationFormation/blob/master/docs/access-denied.md
WARN: AccessDenied: unable to log into account 758169039132. This might have various causes, to troubleshoot:
https://github.com/OlafConijn/AwsOrganizationFormation/blob/master/docs/access-denied.md
ERROR: Task OrganizationUpdate print failed. reason: Cannot read properties of undefined (reading 'serviceControlPolicies')
Cannot read properties of undefined (reading 'serviceControlPolicies') (use option --print-stack to print stack)
ERROR:
ERROR: ==========================
ERROR: Stopped performing task(s)
ERROR: Following tasks failed:
ERROR:  - Task OrganizationUpdate
ERROR: Following tasks were not executed:
ERROR:  - Task OrganizationBuildPipeline
ERROR: ==========================
ERROR:
ERROR: Task OrganizationBuild print failed. reason: Number of failed tasks 1 exceeded tolerance for failed tasks 0.
WARN:
WARN: ========================
WARN: Done performing task(s): 3 failed but did not exceed tolerance for failed tasks 99
WARN: Following tasks failed:
WARN:  - Task OrganizationBuild
WARN:  - Task Types
WARN:  - Task AWSSSO
WARN: ========================
WARN:
(ins)hendry-tw-mbp~/sorg/organization-formation$ aws sts get-caller-identity
{
    "UserId": "705671790868",
    "Account": "705671790868",
    "Arn": "arn:aws:iam::705671790868:root"
}

OlafConijn commented 2 years ago

i think there is two issues here:

you are not able to assume a role (because you are logged in as root). I'll add that to the documentation that is linked in the error. the solution for this is to be logged in using an IAM user or role (be it through SSO or otherwise).
something with serviceControlPolicies.

could you try running the command again with --print-stack? i assume you didn't change the state.json file stored in s3

kaihendry commented 2 years ago

(ins)hendry-tw-mbp~/sorg/organization-formation$ npx org-formation print-stack ./organization-tasks.yml --output yaml --max-concurrent-stacks 100 --max-concurrent-tasks 100
(ins)hendry-tw-mbp~/sorg/organization-formation$ echo $?
0

OlafConijn commented 2 years ago

org-formation perform-tasks ./organization-tasks.yml --no-color --state-bucket-name organization-formation-705671790868 --state-object state.json --print-stack

OlafConijn commented 2 years ago

--print-stack prints the stacktrace of any error to the output. I do realize (only now?) that the term stack got overloaded quite a bit :D

kaihendry commented 2 years ago

(ins)hendry-tw-mbp~/sorg/organization-formation$ npx org-formation perform-tasks ./organization-tasks.yml --no-color --state-bucket-name organization-formation-705671790868 --state-object state.json --print-stack
INFO: Executing: include 000-organization-build/organization-tasks.yml.
INFO: Executing: update-organization organization.yml.
ERROR: Task OrganizationUpdate execute failed. reason: Cannot read properties of undefined (reading 'serviceControlPolicies')
Cannot read properties of undefined (reading 'serviceControlPolicies')
TypeError: Cannot read properties of undefined (reading 'serviceControlPolicies')
    at TaskProvider.createOrganizationalUnitDeleteTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/org-binder/org-tasks-provider.js:305:55)
    at OrganizationBinder.enumBuildTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/org-binder/org-binder.js:93:50)
    at UpdateOrganizationCommand.performCommand (/usr/local/lib/node_modules/aws-organization-formation/dist/src/commands/update-organization.js:50:30)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Function.Perform (/usr/local/lib/node_modules/aws-organization-formation/dist/src/commands/update-organization.js:17:9)
    at async UpdateOrganizationTask.innerPerform (/usr/local/lib/node_modules/aws-organization-formation/dist/src/build-tasks/tasks/organization-task.js:48:9)
    at async UpdateOrganizationTask.perform (/usr/local/lib/node_modules/aws-organization-formation/dist/src/build-tasks/tasks/organization-task.js:38:9)
    at async Function.performTask (/usr/local/lib/node_modules/aws-organization-formation/dist/src/core/generic-task-runner.js:155:17)
    at async Promise.all (index 0)
    at async Function.RunTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/core/generic-task-runner.js:73:17)
    at async Function.RunTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/build-tasks/build-runner.js:13:9)
    at async Object.perform (/usr/local/lib/node_modules/aws-organization-formation/dist/src/build-tasks/tasks/include-task.js:35:17)
    at async Function.performTask (/usr/local/lib/node_modules/aws-organization-formation/dist/src/core/generic-task-runner.js:155:17)
    at async Promise.all (index 0)
    at async Function.RunTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/core/generic-task-runner.js:73:17)
    at async Function.RunTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/build-tasks/build-runner.js:13:9)
ERROR:
ERROR: ==========================
ERROR: Stopped performing task(s)
ERROR: Following tasks failed:
ERROR:  - Task OrganizationUpdate
ERROR: Following tasks were not executed:
ERROR:  - Task OrganizationBuildPipeline
ERROR: ==========================
ERROR:
ERROR: Task OrganizationBuild execute failed. reason: Number of failed tasks 1 exceeded tolerance for failed tasks 0.
ERROR:
ERROR: ==========================
ERROR: Stopped performing task(s)
ERROR: Following tasks failed:
ERROR:  - Task OrganizationBuild
ERROR: Following tasks were not executed:
ERROR:  - Task Types
ERROR:  - Task AWSSSO
ERROR: ==========================
ERROR:
ERROR: Number of failed tasks 1 exceeded tolerance for failed tasks 0.

kaihendry commented 2 years ago

Deleting state.json managed to fix the situation after resetting organization.yml back to basics.

k-paulius commented 1 year ago

I just encountered this same issue.

My starting point was a brand new AWS Org with the following structure

Root
- Deployments
- Prod
  - org-formation-cicd-prod (account)
- management (account)

I ran "org-formation init-pipeline" with org-formation-cicd-prod account being my build account and everything was configured successfully. I then modified organization.yml, but build process failed because I must have made some mistakes in the file.

INFO: Executing: include 000-organization-build/organization-tasks.yml.
INFO: Executing: update-organization organization.yml.
OC::ORG::OrganizationalUnit   | ProdOU                        | Detach Account (OrgFormationCicdProdAccount)
OC::ORG::OrganizationalUnit   | ProdOU                        | Delete
OC::ORG::OrganizationalUnit   | DeploymentsOU                 | Detach OU (ProdOU)
OC::ORG::OrganizationalUnit   | DeploymentsProdOU             | Create (ou-s083-ekpnmrka)
ERROR: failed executing task: Attach OU (DeploymentsProdOU) OC::ORG::OrganizationalUnit DeploymentsOU OrganizationalUnitNotFoundException: You specified an organizational unit that doesn't exist
ERROR: Task OrganizationUpdate execute failed. reason: You specified an organizational unit that doesn't exist
You specified an organizational unit that doesn't exist (use option --print-stack to print stack)

org-formation deleted Deployments -> Prod OU, but it did not update the state.json file. Every run after this event was getting "Cannot read property 'serviceControlPolicies' of undefined" error

INFO: Executing: include 000-organization-build/organization-tasks.yml.
INFO: Executing: update-organization organization.yml.
ERROR: Task OrganizationUpdate execute failed. reason: Cannot read property 'serviceControlPolicies' of undefined
Cannot read property 'serviceControlPolicies' of undefined
TypeError: Cannot read property 'serviceControlPolicies' of undefined
    at TaskProvider.createOrganizationalUnitDeleteTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/org-binder/org-tasks-provider.js:527:55)
    at OrganizationBinder.enumBuildTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/org-binder/org-binder.js:82:50)
    at UpdateOrganizationCommand.performCommand (/usr/local/lib/node_modules/aws-organization-formation/dist/src/commands/update-organization.js:51:30)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async Function.Perform (/usr/local/lib/node_modules/aws-organization-formation/dist/src/commands/update-organization.js:17:9)
    at async UpdateOrganizationTask.innerPerform (/usr/local/lib/node_modules/aws-organization-formation/dist/src/build-tasks/tasks/organization-task.js:48:9)
    at async UpdateOrganizationTask.perform (/usr/local/lib/node_modules/aws-organization-formation/dist/src/build-tasks/tasks/organization-task.js:38:9)
    at async Function.performTask (/usr/local/lib/node_modules/aws-organization-formation/dist/src/core/generic-task-runner.js:155:17)
    at async Promise.all (index 0)
    at async Function.RunTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/core/generic-task-runner.js:73:17)
    at async Function.RunTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/build-tasks/build-runner.js:13:9)
    at async Object.perform (/usr/local/lib/node_modules/aws-organization-formation/dist/src/build-tasks/tasks/include-task.js:35:17)
    at async Function.performTask (/usr/local/lib/node_modules/aws-organization-formation/dist/src/core/generic-task-runner.js:155:17)
    at async Promise.all (index 0)
    at async Function.RunTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/core/generic-task-runner.js:73:17)
    at async Function.RunTasks (/usr/local/lib/node_modules/aws-organization-formation/dist/src/build-tasks/build-runner.js:13:9)

Removing the information about, non-existant, Prod OU from the state.json file solved the issue.

sshvetsov commented 1 year ago

@OlafConijn, I think the changes you've introduced in the 1.0.10-beta2 release may close this issue.

org-formation / org-formation-cli