aws-ia / terraform-aws-control_tower_account_factory

AWS Control Tower Account Factory
Apache License 2.0
640 stars 442 forks source link

[ERROR] Account ID Not Available was not found in the Organization #39

Closed lliknart closed 2 years ago

lliknart commented 2 years ago

Hi, AFT is deployed and I already used it to create new AWS accounts and enroll them in a specific OU.

But from today, when I try to create a new account in another OU, I have an error and no accounts are created

Everything starts fine:

aft-account-request-processor lambda log

{
"time_stamp": "2022-01-21 12:04:12,161",
"log_level": "INFO",
"log_message": "There are messages pending processing"
}

{
"time_stamp": "2022-01-21 12:04:12,163",
"log_level": "INFO",
"log_message": "Validating new CT Account Request"
}

{
"time_stamp": "2022-01-21 12:04:32,069",
"log_level": "INFO",
"log_message": "Creating new account leveraging parameters: [{'Key': 'SSOUserEmail', 'Value': 'my-sso-email@my_company.com'}, {'Key': 'AccountEmail', 'Value': 'my-email@my_company.com'}, {'Key': 'SSOUserFirstName', 'Value': 'Bob'}, {'Key': 'SSOUserLastName', 'Value': 'Patrick'}, {'Key': 'ManagedOrganizationalUnit', 'Value': 'My_OU (ou-my_ou_id)'}, {'Key': 'AccountName', 'Value': 'My-Account-Name}]"
}

But here is the error aft-invoke-aft-account-provisioning-framework lambda log

{
"time_stamp": "2022-01-21 12:06:51,301",
"log_level": "INFO",
"log_message": "Getting account email for account id Not Available"
}

{
    "time_stamp": "2022-01-21 12:06:51,579",
    "log_level": "ERROR",
    "log_message": {
        "FILE": "aft_invoke_aft_account_provisioning_framework.py",
        "METHOD": "lambda_handler",
        "EXCEPTION": "Account ID Not Available was not found in the Organization"
    }
}

Traceback (most recent call last):
File "/var/task/aft_invoke_aft_account_provisioning_framework.py", line 93, in lambda_handler
invoke_event = build_invoke_event(
File "/var/task/aft_invoke_aft_account_provisioning_framework.py", line 54, in build_invoke_event
account_email = utils.get_account_email_from_id(
File "/opt/python/lib/python3.8/site-packages/aft_common/aft_utils.py", line 551, in get_account_email_from_id
raise Exception("Account ID " + id + " was not found in the Organization")
Exception: Account ID Not Available was not found in the Organization

What could be the reason ?

mathewmoon commented 2 years ago

@lliknart Is this OU nested or directly under the root OU?

lliknart commented 2 years ago

The parent OU of My_OU is the root OU

mathewmoon commented 2 years ago

I believe the syntax you are using for the OU is reserved for nesting based on the docs.

balltrev commented 2 years ago

@mathewmoon is correct here. @lliknart, can you update the value to only "My_OU" and see if the account request completes?

lliknart commented 2 years ago

Ok, hereafter actions I made:

At first, I created OU through Control Tower Console: Root |------Security |------Sandbox |------My_OU |------Other_OU

When I use AFT to create Account in Sandbox OU, I used this syntax:

control_tower_parameters = {
   [...]
    ManagedOrganizationalUnit = "Sandbox (ou-exct-sandbox-OU-Id)"
   [...]
  }

And it worked. So I used the same syntax for My_OU and Other_OU

    ManagedOrganizationalUnit = "My_OU (ou-exct-My-OU-Id)"

It produced errors shown above.

Then I try to to do what you suggest. I have now this error from aft_account_request_processor lambda:

{
    "time_stamp": "2022-01-21 22:24:13,335",
    "log_level": "ERROR",
    "log_message": {
        "FILE": "aft_account_request_processor.py",
        "METHOD": "lambda_handler",
        "EXCEPTION": "An error occurred (InvalidParametersException) when calling the GetProvisionedProductOutputs operation: Last Successful Provisioning Record doesn't exist."
    }
}
Traceback (most recent call last):
  File "/var/task/aft_account_request_processor.py", line 215, in lambda_handler
    modify_existing_account(
  File "/var/task/aft_account_request_processor.py", line 136, in modify_existing_account
    product_outputs_response = client.get_provisioned_product_outputs(
  File "/opt/python/lib/python3.8/site-packages/botocore/client.py", line 386, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/opt/python/lib/python3.8/site-packages/botocore/client.py", line 705, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidParametersException: An error occurred (InvalidParametersException) when calling the GetProvisionedProductOutputs operation: Last Successful Provisioning Record doesn't exist.

So, I decided to delete those file from the aft-account-request repository to delete those entries from DynamoDB in gitops way... I try again with the syntax you suggest but now I have this error:

{
    "time_stamp": "2022-01-21 22:45:59,897",
    "log_level": "ERROR",
    "log_message": {
        "FILE": "aft_account_request_processor.py",
        "METHOD": "lambda_handler",
        "EXCEPTION": "An error occurred (InvalidParametersException) when calling the ProvisionProduct operation: A stack named My_Account_Name already exists."
    }
}
Traceback (most recent call last):
  File "/var/task/aft_account_request_processor.py", line 209, in lambda_handler
    response = create_new_account(
  File "/var/task/aft_account_request_processor.py", line 87, in create_new_account
    response = client.provision_product(
  File "/opt/python/lib/python3.8/site-packages/botocore/client.py", line 386, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/opt/python/lib/python3.8/site-packages/botocore/client.py", line 705, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidParametersException: An error occurred (InvalidParametersException) when calling the ProvisionProduct operation: A stack named My_Account_Name already exists.

What is the best way to do what I want to do ?

Edit: I tried with another account name on My_OU with syntax you suggest, and I have the same error

{
    "time_stamp": "2022-01-22 15:35:16,943",
    "log_level": "ERROR",
    "log_message": {
        "FILE": "aft_invoke_aft_account_provisioning_framework.py",
        "METHOD": "lambda_handler",
        "EXCEPTION": "Account ID Not Available was not found in the Organization"
    }
}

Edit 2: I can't even create new account in Sandbox OU. Do you know where I can find the created stack ? because when I try via AWS Control Tower Console to enroll an account, it tells me the stack alreasy exists but I can't find it (I looked through ServiceCatalog, CloudFormation, Control Tower)

hanafya commented 2 years ago

This will require a deeper dive here, can you go ahead and open a support case with AWS Premium Support to help further troubleshoot this issue?

mathewmoon commented 2 years ago

@hanafya I think the syntax for specifying a nested OU leaves some room for improvement. FWIW I think using a more canonical way of specifying the target OU and parent would likely prevent a lot of future tickets. For instance:

ManagedOrganizationalUnit = "child-ou-id"
ParentOrganizationalUnit = "parent-id"
lliknart commented 2 years ago

Hi, I already created a case with AWS Premium Support. Apparently, I have reached the limit on the number of accounts in my organization. But I couldn't find this error. It could be great if AFT module raises this error in the appropriate Lambda log instead of Account ID Not Available was not found in the Organization

The error support raises me: AWS Control Tower cannot create an account because you have reached the limit on the number of accounts in your organization

It's weird I couldn't find it through AWS Console. I will check with support.

Do I let this issue open to request logging improvement ?

stumins commented 2 years ago

Hi, I'm glad you were able to resolve the issue. Thanks for letting us know what the root cause was.

We'll add a backlog item to improve logging around this case. I'm going to close this issue, please feel free to open another issue if you run into any additional problems.