DataDog / datadog-cloudformation-resources

Apache License 2.0
49 stars 35 forks source link

Consider supporting log ingest enablement through a resource like Datadog::Integrations::AWS #224

Open dls314 opened 1 year ago

dls314 commented 1 year ago

Note: If you have a feature request, you should contact support so the request can be properly tracked.

Is your feature request related to a problem? Please describe.

I would like to use a resource like Datadog::Integrations::AWS resources to enable log collection for the account within the AWS integration for the automatically set-up triggers referenced here: https://docs.datadoghq.com/logs/guide/send-aws-services-logs-with-the-datadog-lambda-function/?tab=awsconsole#automatically-set-up-triggers

Describe the solution you'd like

Create a resource like the Datadog::Integrations::AWS resource and examples that use it to allow specifying the collection of logs from sources with automatically set up triggers; including: API Gateway Access Logs, API Gateway Execution Logs, Application ELB Access Logs, Classic ELB Access Logs, CloudFront Access Logs, Lambda Logs, Redshift Logs, and S3 Access Logs.

A specification mechanism like the one for AccountSpecificNamespaceRules in the the Datadog::Integrations::AWS resource could be used to specify each log type in the resource configuration https://github.com/DataDog/datadog-cloudformation-resources/blob/master/datadog-integrations-aws-handler/datadog-integrations-aws.json#L69-L78

Another resource type besides Datadog::Integrations::AWS is likely useful since it will need to specify the ARN of the Datadog forwarder lambda being used.

Describe alternatives you've considered We write our own CFN custom resource provider to handle this today, and roll that custom resource provider out to our accounts. It's a pain, and a maintenance burden that I'd rather not have.

We just end up posting to these Datadog APIs from that resource provider

Additional context

image

github-actions[bot] commented 1 year ago

Thanks for your contribution!

This issue has been automatically marked as stale because it has not had activity in the last 30 days. Note that the issue will not be automatically closed, but this notification will remind us to investigate why there's been inactivity. Thank you for participating in the Datadog open source community.

If you would like this issue to remain open:

  1. Verify that you can still reproduce the issue in the latest version of this project.

  2. Comment that the issue is still reproducible and include updated details requested in the issue template.

dls314 commented 1 year ago

This issue is still reproduceable, and is related to the support request #942037