[AWS] Requests include loggroups from linked accounts

[heisenbergye]

Hi @kaiyan-sheng , nice to see the feature of supporting collect metrics form linkaccount https://github.com/elastic/beats/pull/35540.

And CloudWatch Logs also need this feature, because aws-cloudwatch can not find log groups from the source accounts when using CloudWatch cross-account observability.

{"log.level":"error","@timestamp":"2024-01-21T17:26:53.378Z","log.logger":"input.aws-cloudwatch.cloudwatch_poller","log.origin":{"function":"github.com/elastic/beats/v7/x-pack/filebeat/input/awscloudwatch.(*cloudwatchPoller).run","file.name":"awscloudwatch/cloudwatch.go","file.line":66},"message":"getLogEventsFromCloudWatch failed: error FilterLogEvents with Paginator: operation error CloudWatch Logs: FilterLogEvents, https response error StatusCode: 400, RequestID: 775cd847-41f6-4615-9040-dcc792591455, ResourceNotFoundException: The specified log group does not exist.","service.name":"filebeat","id":"CE9F457D5ABA17C7","ecs.version":"1.6.0"}

DescribeLogGroups API also has provided the parameter includeLinkedAccounts for users to include loggroups from source accounts in a monitoring account.

account-identifiers (list) When includeLinkedAccounts is set to True , use this parameter to specify the list of accounts to search. You can specify as many as 20 account IDs in the array.

https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_DescribeLogGroups.html

thanks a lot

[kaiyan-sheng]

Hi @heisenbergye , this issue should be addressed by https://github.com/elastic/beats/pull/41206. Thanks!

[elasticmachine]

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

[Kavindu-Dodan]

Feature is delivered through PR #41188 & #41206 and will be available from filebeats version 8.16.x onwards.