Open cspring86 opened 3 months ago
I want to confirm, the centralized bucket being used for the logs is a self-built bucket encrypted with CMK, right?
You can search for "KMSPublicAccessPolicy" in IAM -> Policy and add the corresponding CMK permissions.
I will reproduce this issue and fix it in version 2.2.0.
I want to confirm, the centralized bucket being used for the logs is a self-built bucket encrypted with CMK, right?
You can search for "KMSPublicAccessPolicy" in IAM -> Policy and add the corresponding CMK permissions.
Yes, I think I'm going to manually add the permissions for now.
I will reproduce this issue and fix it in version 2.2.0.
Amazing, thanks!
Describe the bug
I've deployed the Light Engine version of the solution exclusively.
I've configured log merge after 7 days and log archive after 365 days.
I've been using Athena to query the log data perfectly fine since the solution has been deployed. However, after a couple of months, I tried querying log data older than 7 days and it returned nothing. So I checked the analytics S3 bucket and confirmed that no data existed older than 7 days.
After investigating, I've found the following error in the
S3ObjectMigration
Lambda logs:The analytics S3 bucket has a customer managed KMS key as the default key, so it can't upload the newly merged object back to the analytics bucket.
Expected Behavior
The solution should support customer managed KMS keys on the analytics S3 bucket such that it accepts the KMS key as a parameter and adds the relevant permissions to the S3ObjectMigration Lambda role.
The solution should also properly fail when the error occurs. Currently, everything looks like it's working perfectly. The state machine all succeed, the Lambdas all succeed, so there's no clear indication something has gone wrong.
Current Behavior
The solution silently fails when uploading merged logs back to the analytics S3 bucket, if that bucket has a customer managed KMS key as its default encryption key.
The solution also reports success when the error occurs in the State Machines and Lambda functions.
Reproduction Steps
Possible Solution
Allow end user to provide a customer managed KMS key to the solution for the analytics S3 bucket so it can update the S3ObjectMigration Lambda with the relevant permissions.
Even better, have the solution check the S3 bucket for default encryption with KMS, and if set, fetch the KMS key information and configure itself accordingly.
Additional Information/Context
No response
Solution Version
2.1.1
AWS Region. e.g., us-east-1
us-east-1
Other information
No response