How to set AWS cloudtrail input from another aws account bucket

[qoovsxp]

Hi,

We follow this document to set sharing CloudTrail Log Files Between AWS Accounts. https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-sharing-logs.html We put B AWS account’s cloudtrail logs to A AWS account bucket. And we let cloudtrail iam user can access A AWS account bucket.

We set a assume role to input try to get log from A AWS account bucket,but a SQS error occur. I guess it's cause by the assume role cant's access the SQS queue on B AWS account. And it occur can’t get log from s3 bucket if we don’t set assume role to input.

Anyone can help us?

Thanks.

[radykal-com]

Hello,

what version of the graylog-aws-plugin are you using and how is your input configured?

[qoovsxp]

Hi radykal,

My graylog version is 2.4.4 and that has integrate graylog-aws-plugin. I set the assume role on input.

The sqs set on AWS as follow.

[radykal-com]

Well, looks like some kind of permission problems. Can you describe what resources and roles(with permissions) are in each account?

[qoovsxp]

A acccount: s3==> I build a s3 bucket named "sanderson-cloudtrail" and set the policy for receive cloudtrail from B account.And sure could receive cloudtrail logs on s3 bucket.

https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-set-bucket-policy-for-multiple-accounts.html

IAM role==> I set a IAMRole named "sanderson-readlog" for Cross-Account Access and give s3 and SQS permission.

B account: IAM User==> I create a iam user for read sqs and then attach administrator and assumerole's permission.

Cloudtrail==>

SNS==>

SQS==> I set a sqs named "sandersontest" for subscribe sns notify from cloudtrail.

I poll messages from sqs queue "sandersontest" and confirmed that direction is right.

These configuration of above will occur error as follow on graylog.

Thank you.

[radykal-com]

Well, your setup looks fine for me. I'll try to configure one of my graylog instances with the same setup and check what happens.

[qoovsxp]

OK，appreciate your help.I am looking forward to hear about your test result.I have stuck at this error about two month and try this lab for GDPR scenario.I think that could help graylog more suitable in enterprise environment if we resolve this error.

[et304383]

Would like support for this too. It's AWS best practice to run multiple accounts and centralize CloudTrail logs into a single bucket in a logging account.

It would be incredibly cumbersome to have each account's CloudTrail send SNS notifications to the logging account.

The CloudTrail plugin should support parsing SQS messages sent by S3 event notifications, not just CloudTrail notifications.

The format is defined here:

https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html

Please add support for this format so we can keep things centralized and not needlessly rely on SNS which just inflates the cost of getting the logs to Graylog for no added value.

[et304383]

@qoovsxp I got around the current limitations by putting S3 events on the CloudTrail bucket, sending them to Lambda, then restructuring them to look like the SNS log delivery messages the plugin wants.

Some python code:

import json

import boto3

sqs_client = boto3.client('sqs')

def handler(event, context):
    s3_event = event['Records'][0]['s3']
    bucket_name = s3_event['bucket']['name']
    object_key = s3_event['object']['key']

    if 'CloudTrail-Digest' in object_key:
        return None

    message = {
        's3Bucket': bucket_name,
        's3ObjectKey': [
            object_key
        ]
    }

    sqs_message = {
        'Message': json.dumps(message)
    }

    sqs_client.send_message(
        QueueUrl='<SQS queue URL>',
        MessageBody=json.dumps(sqs_message)
    )

[naggappan]

Do SNS support cross account? If yes Then I can provide my aws log account SNS topic to all my other accounts cloud trial. @et304383 if I use the above come then in cloud trial of different accounts I don't need to select SNS notification right?