Unable to deploy in Audit Account

kclinden commented 4 months ago

Describe the bug

I am trying to deploy this in the Control Tower created Audit account which is already configured as a subscriber in Security Lake. I have already configured a Lakeformation and Tables and have been using Athena to query it so this is validated to be working. When deploying the CloudFormation Stack I am getting an error on the CreateLakeFormationPermissionsCustomResource resource, but the logging doesn't tell me what the issue is specifically.

CloudFormation Events Error

Received response status [FAILED] from custom resource. Message returned: See the details in CloudWatch Log Stream: 2024/04/12/[$LATEST]37531d6f5c56408d98a8cc826a79be5f (RequestId: b6d65d04-c560-4194-976f-658a0fd33027)

When looking further into this I had started by looking at the CloudWatch logs as mentioned in the CFT error, but that didn't have much information even when Lambda was in Debug mode. I eventually found a CloudTrail API for lakeformation that gave me the following error. This seems like it is trying to modify the AmazonSecurityLakeMetaStoreManager permissions, but when using v2 this is actually renamed to AmazonSecurityLakeMetaStoreManagerv2 and the non-v2 role is removed as documented here.

    "errorCode": "InvalidInputException",
    "errorMessage": "Invalid principal, arn: arn:aws:iam::123456789000:role/service-role/AmazonSecurityLakeMetaStoreManager",
    "requestParameters": {
        "catalogId": "123456789000",
        "dataLakeSettings": {
            "dataLakeAdmins": [
                {
                    "dataLakePrincipalIdentifier": "arn:aws:iam::123456789000:role/service-role/AmazonSecurityLakeMetaStoreManager"
                },
                {
                    "dataLakePrincipalIdentifier": "arn:aws:iam::123456789000:role/aws-reserved/sso.amazonaws.com/AWSReservedSSO_AWSAdministratorAccess_e1fa40dceeb3188a"
                }
            ],
            "readOnlyAdmins": [],
            "createDatabaseDefaultPermissions": [],
            "createTableDefaultPermissions": [],
            "parameters": {
                "CROSS_ACCOUNT_VERSION": "3"
            },
            "allowExternalDataFiltering": false,
            "whitelistedForExternalDataFiltering": [],
            "externalDataFilteringAllowList": [],
            "disallowGrantOnIAMAllowedPrincipals": false
        }
    }

To Reproduce

Deploy without v1 metadata role existing.

Expected behavior Stack deploys properly

[x] Version: v1.0.0
[x] Region: us-east-1
[x] Was the solution modified from the version published on this repository? no
[x] If the answer to the previous question was yes, are the changes available on GitHub?
[x] Provide the logs by enabling the debug mode in the hub template.

Additional context Above

deolank commented 4 months ago

Hi, thanks for your feedback. We will improve the logging in the next release. For the root cause, this lambda function gets the list of existing admins on the LakeFormation which seems to have the old IAM role. When the IAM role was deleted, did you also remove it from the 'Data lake administrators' section in the LakeFormation console? If not, please delete the old IAM role from the list of admins and deploy the solution again.

kclinden commented 4 months ago

When I had it deployed it looks like this creates a new lake formation in the audit account and it doesn't use the one that I have already configured. Security Lake is in a different account (Log Archive).

The arn arn:aws:iam::123456789000:role/service-role/AmazonSecurityLakeMetaStoreManager doesn't exist in the Audit account.

kclinden commented 4 months ago

I found what you were talking about and removed it. Now I got a new error :(

    "errorCode": "AccessDenied",
    "errorMessage": "An unknown error occurred",
    "requestParameters": {
        "catalogId": "<log_archive_account>",
        "databaseName": "aws_solutions_resource_link_database",
        "tableInput": {
            "name": "amazon_security_lake_table_us_east_1_vpc_flow_2_0",
            "retention": 0,
            "targetTable": {
                "catalogId": "<audit_account_id>",
                "databaseName": "amazon_security_lake_glue_db_us_east_1",
                "name": "amazon_security_lake_table_us_east_1_vpc_flow_2_0"
            },
            "isRowFilteringEnabled": false
        }
    },

kclinden commented 4 months ago

I am using IAM Identity Center auth with Quicksight. Would that cause this issue? I can't find any cloudtrail logs that give me much info on the access denied error.

deolank commented 4 months ago

Yes, if you are using Identity Center to manage QuickSight, the create_group API for User group creation will return 'Access Denied' error and the deployment will fail. We are planning to do a release to address this soon.

kubernetes-repo-addons commented 3 months ago

Any updates on audit account deployment ?

deolank commented 3 months ago

We have released the new version which fixes this issue. During deployment, select "No" as the option for input parameter CreateQuickSightUserGroups. Please deploy the latest template to resolve the issue.

aws-solutions / security-insights-on-aws

Unable to deploy in Audit Account #2