terraform-aws-modules / terraform-aws-datadog-forwarders

Terraform module to create resources on AWS to forward logs/metrics to Datadog 🇺🇦
https://registry.terraform.io/modules/terraform-aws-modules/datadog-forwarders/aws
Apache License 2.0
56 stars 41 forks source link

Not forwarding S3 logs to Datadog? #40

Open Darwiner opened 5 months ago

Darwiner commented 5 months ago

Description

This might be halfway between a "bug" (because of the description) and a feature request at the same time...

The s3_log_bucket_arns variable in https://github.com/terraform-aws-modules/terraform-aws-datadog-forwarders/tree/master/modules/log_forwarder has a description that mentions:

S3 log buckets for forwarder to read and forward logs to Datadog

...yet, looking through the code (and also trying it out), is it possible that it does not take care of adding the actual forwarding of logs to Datadog?

I was expecting to see a aws_s3_bucket_notification resource, which I believe would have taken care of this part, but there does not seem to be one? Or might I be missing something?

Versions

Expected behavior

Lambda function being aware of new logs in S3 bucket and forwarding those to Datadog. Lambda logs/metrics showing activity. New logs appearing in Datadog.

Actual behavior

S3 logs are not getting forwarded by the lambda function. Lambda function logs are empty. Nothing appearing in Datadog logs.

tennengabr commented 3 months ago

@Darwiner I was also confused by this as was brought into the implementation half way through process. After reading the docs on which the module is based on (datadog-serverless-functions/aws/log_monitoring).

It appears you can either create the triggers manually using the variables exported from this module or Datadog will create them for your automatically when you setup your Datadog integrations.

Here is more information on setting up the triggers using automatic/manual approaches. Send AWS Services Logs With The Datadog Lambda Function - Set up triggers