Not able to add existing accounts when deploying Conformity stack

[xu-lei-richard]

Environments:

Existing AWS Control Tower enabled multi-accounts environment, Subscribed to the Conformity through AWS Market place, Sign in to our AWS Control Tower management account as an administrator, launched the AWS CloudFormation template. The stack was created successfully.

Issue

Existing accounts are not shown in our Conformity service. Looked through CloudWatch logs, and discovered errors in adding account.

Error log 1: In the main management account:

[INFO] 2021-07-10T11:58:05.992Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Event received by handler: {'InvokeAction': 'configure_account', 'account_id': '123456789515'} [INFO] 2021-07-10T11:58:05.992Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b function name: trend-micro-cloud-one-confor-LifecycleEventHandler-4j63YSPDjNpx invoked arn: arn:aws:lambda:ap-southeast-2:123456789515:function:trend-micro-cloud-one-confor-LifecycleEventHandler-4j63YSPDjNpx [INFO] 2021-07-10T11:58:06.024Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Found credentials in environment variables. [INFO] 2021-07-10T11:58:06.977Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Create Connector Object [INFO] 2021-07-10T11:58:06.977Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Create role in target account [INFO] 2021-07-10T11:58:06.980Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Retrieving session for operation [INFO] 2021-07-10T11:58:07.809Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b currently executing in 123456789515; called account is 123456789515 [INFO] 2021-07-10T11:58:08.014Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Target account is Control Tower Management; returning local credentials session [INFO] 2021-07-10T11:58:08.038Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Found credentials in environment variables. [INFO] 2021-07-10T11:58:08.909Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Creating role CloudOneConformityConnectorRole and policy CloudOneConformityConnectorPolicy in account 123456789515 [INFO] 2021-07-10T11:58:08.910Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Creating role... [INFO] 2021-07-10T11:58:09.167Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b http status for call to /organisation/external-id was: 403 with response: b'{ "Message": "User is not authorized to access this resource with an explicit deny" }' [ERROR] 2021-07-10T11:58:09.168Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Failed to create role: 'data' [ERROR] 2021-07-10T11:58:09.172Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Failed to configure account 123456789515 with exception: 'data' [INFO] 2021-07-10T11:58:29.194Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Add account to Cloud One Conformity [INFO] 2021-07-10T11:58:29.401Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b http status for call to /accounts was: 403 with response: b'{ "Message": "User is not authorized to access this resource with an explicit deny" }' [INFO] 2021-07-10T11:58:29.404Z 38c1c6ea-9ac2-455d-ab4c-2ea6b7cfc48b Done

Error log 2: In the non-management account:

[INFO] 2021-07-10T11:58:05.854Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e Event received by handler: {'InvokeAction': 'configure_account', 'account_id': '123456789137'} [INFO] 2021-07-10T11:58:05.854Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e function name: trend-micro-cloud-one-confor-LifecycleEventHandler-4j63YSPDjNpx invoked arn: arn:aws:lambda:ap-southeast-2:123456789515:function:trend-micro-cloud-one-confor-LifecycleEventHandler-4j63YSPDjNpx [INFO] 2021-07-10T11:58:05.889Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e Found credentials in environment variables. [INFO] 2021-07-10T11:58:06.885Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e Create Connector Object [INFO] 2021-07-10T11:58:06.885Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e Create role in target account [INFO] 2021-07-10T11:58:06.888Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e Retrieving session for operation [INFO] 2021-07-10T11:58:07.700Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e currently executing in 123456789515; called account is 123456789137 [INFO] 2021-07-10T11:58:08.345Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e Assumed session for 123456789137 - AWSControlTowerExecution. [INFO] 2021-07-10T11:58:09.238Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e Creating role CloudOneConformityConnectorRole and policy CloudOneConformityConnectorPolicy in account 123456789137 [INFO] 2021-07-10T11:58:09.238Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e Creating role... [INFO] 2021-07-10T11:58:09.440Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e http status for call to /organisation/external-id was: 403 with response: b'{ "Message": "User is not authorized to access this resource with an explicit deny" }' [ERROR] 2021-07-10T11:58:09.440Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e Failed to create role: 'data' [ERROR] 2021-07-10T11:58:09.443Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e Failed to configure account 123456789137 with exception: 'data' [INFO] 2021-07-10T11:58:29.464Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e Add account to Cloud One Conformity [INFO] 2021-07-10T11:58:29.601Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e http status for call to /accounts was: 403 with response: b'{ "Message": "User is not authorized to access this resource with an explicit deny" }' [INFO] 2021-07-10T11:58:29.603Z a069d539-4dd5-4ec2-9b90-ac7ead5e4c2e Done

Same error when Creating role CloudOneConformityConnectorRole and policy CloudOneConformityConnectorPolicy in account.

Doubled checked all accounts that the AWSControlTowerExecution roles do have full AWS admin permissions.

Note: Part of the account number has been changed for privacy reason.

[kkvinjam]

Since the message indicated explicit deny. The error looks like an SCP(Service Control Policy) in your AWS Organization is denying iam:CreateRole. Please check your SCPs.

[xu-lei-richard]

Thanks for your reply, @kkvinjam

I double checked SCPs and don't think this is caused by iam:CreateRole related policies. My reasons are:

• The error message is about "http status for call to /organisation/external-id", with 403 forbidden, although this is part of creating role CloudOneConformityConnectorRole. But it was not creating a role.
• The environment is a newly established AWS Control Tower setup, with standard guardrails. Nothing custom has been deployed. This solution is designed for Control Tower Integration, isn't it?
• Tested with root account, with the same error.
• The only iam:CreateRole related policy part is: { "Condition": { "ArnNotLike": { "aws:PrincipalArn": [ "arn:aws:iam::*:role/AWSControlTowerExecution", "arn:aws:iam::*:role/stacksets-exec-*" ] } }, "Action": [ "iam:AttachRolePolicy", "iam:CreateRole", "iam:DeleteRole", "iam:DeleteRolePermissionsBoundary", "iam:DeleteRolePolicy", "iam:DetachRolePolicy", "iam:PutRolePermissionsBoundary", "iam:PutRolePolicy", "iam:UpdateAssumeRolePolicy", "iam:UpdateRole", "iam:UpdateRoleDescription" ], "Resource": [ "arn:aws:iam::*:role/aws-controltower-*", "arn:aws:iam::*:role/*AWSControlTower*", "arn:aws:iam::*:role/stacksets-exec-*" ], "Effect": "Deny", "Sid": "GRIAMROLEPOLICY" }

To be created role's name is CloudOneConformityConnectorRole, that doesn't match to the resource in the policy.

I think I have missed something. But triple checked deployment guide, I cannot locate anything wrong in my steps. Hope someone knows what it is happening. Thanks!

[TomRyan-321]

Hey @xu-lei-richard, when you are getting denies on retrieving the org external ID it typically is because of either of the following: 1. Invalid API Key, or 2. your conformity region isn't set correctly in the stack. All Conformity accounts accessed via Cloud One are in the us-west-2 region, where standalone accounts can be any of the following in ap-southeast-2 (Sydney), eu-west-1 (Ireland), us-west-2 as per cloud one.

[xu-lei-richard]

Thanks @TomRyan-321 I think it could be the issue. Our account uses ap-southeast-2. I will test again.