Support Kinesis Firehose forwarder

[JayFields]

I use Kinesis Firehose instead of a lambda to forward logs to Datadog. I am unable to use the macro in SAM because it doesn't support adding a roleArn:

Error: Failed to create changeset for the stack: my-stack ex: Waiter ChangeSetCreateComplete failed:
Waiter encountered a terminal failure state Status: FAILED.
Reason: Transform xxxxxxxxxxxx::DatadogServerless failed with:
destinationArn for vendor firehose cannot be used without roleArn

It would be great to support Kinesis Firehose too. Thanks!

[tianchu]

@JayFields Sorry for the late response, just to clarify, you ONLY want logs? If so, you need to manage the subscription of kinesis firehose on log groups on your own using SAM/CloudFormation resources. If you do need enhanced metrics and traces from your Lambda functions as well, then you need to use the Datadog Forwarder Lambda function instead of the Kinesis firehose.

[JayFields]

@tianchu sorry for the late response too. My ultimate goal was to be able to use the datadog-lambda python library to send some basic metrics (counts) from my SAM lambdas to Datadog. The docs recommend using this macro to set it up. However, I want to use Kinesis instead of the Forwarder Lambda. But, as you say, it appears that isn't supported. My goal was to send metrics with zero overhead and without launching a bunch of Forwarder Lambda instances. The MONITORING log line actually works really well for this over Kinesis but it's deprecated.

[tianchu]

@JayFields you can try our Lambda extension https://docs.datadoghq.com/serverless/datadog_lambda_library/extension if you happen to be using Python or Node.js. The extension will aggregate and send metrics directly to the Datadog intake (i.e., no need for the Forwarder).

BTW, I'm going to close this issue, since it's not a direct problem in the forwarder code. For further help or feedback, you can find us (or DM me) in #serverless channel once you join the Datadog Slack community.