Amazon Security Lake integration permission denied on Lambda function

[AlexRuiz7]

Description

During the internal testing of the Amazon Security Lake integration, it was reported a failure on the AWS Lambda function caused by Access Denied error.

START RequestId: 2a5345ff-6116-422d-a858-713b9216cc86 Version: $LATEST
[ERROR] 2024-05-28T12:15:23.482Z    2a5345ff-6116-422d-a858-713b9216cc86    Failed to read S3 object sample.txt from bucket wazuh-aws-security-lake-raw: An error occurred (AccessDenied) when calling the GetObject operation: Access Denied
LAMBDA_WARNING: Unhandled exception. The most likely cause is an issue in the function code. However, in rare cases, a Lambda runtime update can cause unexpected function behavior. For functions using managed runtimes, runtime updates can be triggered by a function change, or can be applied automatically. To determine if the runtime has been updated, check the runtime version in the INIT_START log entry. If this error correlates with a change in the runtime version, you may be able to mitigate this error by temporarily rolling back to the previous runtime version. For more information, see https://docs.aws.amazon.com/lambda/latest/dg/runtimes-update.html
[ERROR] IndexError: list index out of range
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 170, in lambda_handler
    parquet_key = get_full_key(src_location, account_id, region, key, 'parquet')
  File "/var/task/lambda_function.py", line 112, in get_full_key
    filename = ''.join(filename_parts[2].split('-'))END RequestId: 2a5345ff-6116-422d-a858-713b9216cc86
REPORT RequestId: 2a5345ff-6116-422d-a858-713b9216cc86  Duration: 201.11 ms Billed Duration: 202 ms Memory Size: 512 MB Max Memory Used: 162 MB Init Duration: 1350.08 ms

Tasks

• [x] Investigate why this is happening, as the User running the Lambda is the owner of the S3 Bucket.
• [x] Improve the code to handle that exception.

[AlexRuiz7]

During the AWS Lambda configuration, it's needed to grant permissions to the Lambda function to access the S3 bucket, as seen in this guide. That solves the issue.

[AlexRuiz7]

Improved logging

[INFO]  2024-05-29T10:18:36.787Z        Found credentials in environment variables.
START RequestId: 220ef125-3f94-43f7-a2e2-c6394f49185d Version: $LATEST
[INFO]  2024-05-29T10:18:36.954Z    220ef125-3f94-43f7-a2e2-c6394f49185d    Lambda function invoked due to 20240422_ls.s3.2f062956-5a30-4c2a-b693-a0f5d878294c.2024-04-22T14.20.part39.txt.
[INFO]  2024-05-29T10:18:36.954Z    220ef125-3f94-43f7-a2e2-c6394f49185d    Source bucket name is REDACTED. Destination bucket is REDACTED.
[INFO]  2024-05-29T10:18:36.954Z    220ef125-3f94-43f7-a2e2-c6394f49185d    Reading 20240422_ls.s3.2f062956-5a30-4c2a-b693-a0f5d878294c.2024-04-22T14.20.part39.txt.
[INFO]  2024-05-29T10:18:37.125Z    220ef125-3f94-43f7-a2e2-c6394f49185d    Transforming Wazuh security events to OCSF.
[INFO]  2024-05-29T10:18:37.311Z    220ef125-3f94-43f7-a2e2-c6394f49185d    Uploading data to REDACTED.
END RequestId: 220ef125-3f94-43f7-a2e2-c6394f49185d
REPORT RequestId: 448ff821-292b-4e61-b788-f4f59d0ed68d  Duration: 354.24 ms Billed Duration: 355 ms Memory Size: 512 MB Max Memory Used: 168 MB