Implement Datadog client side telemetry

[DerGut]

When sending large volumes of metrics to a single agent, a number of issues can occur. The Datadog docs provide some help for tuning OS specific parameters when they do. The document also mentions a number of client side metrics that are implemented by other dogStatsD clients which can help troubleshoot such issues.

For instance, in https://github.com/brightcove/hot-shots/issues/131 users of hot-shots have encountered Error sending hot-shots message: Error: congestion (exposed by the errorHandler) when using the UDS protocol. This error indicates a problem that could be easily debugged (or at least pointed out) with such metrics without propagating an error to the user. Especially since the the UDS implementation should mimic the behaviour of UDP and should therefore silently drop packages (UDS support specification).

The following metrics are currently implemented by other clients (not all of the clients implement all of the metrics):

• [ ] datadog.dogstatsd.client.metrics
• [ ] datadog.dogstatsd.client.events
• [ ] datadog.dogstatsd.client.service_checks
• [ ] datadog.dogstatsd.client.bytes_sent
• [ ] datadog.dogstatsd.client.bytes_dropped
• [ ] datadog.dogstatsd.client.bytes_dropped_queue
• [ ] datadog.dogstatsd.client.bytes_dropped_writer
• [ ] datadog.dogstatsd.client.packets_sent
• [ ] datadog.dogstatsd.client.packets_dropped
• [ ] datadog.dogstatsd.client.packets_dropped_queue
• [ ] datadog.dogstatsd.client.packets_dropped_writer
• [ ] datadog.dogstatsd.client.bytes_dropped_writer
• [ ] datadog.dogstatsd.client.metric_dropped_on_receive

In addition, the metrics are tagged with client-specific tags. For hot-shots these would be something like:

• [ ] client:js or client:node
• [ ] client_version:<CURRENT_VERSION>
• [ ] client_transport:uds or client_transport:udp

It should also be possible to disable client side telemetry from the options with an argument such as:

• [ ] disableTelemetry=true

[bdeitte]

And as noted in the other ticket, I am definitely in favor of this, and while I won't be able to get to it myself, happy to review or give suggestions as needed.

[cozzbie]

Hi @bdeitte @DerGut. Just wondering if anything has been done about this since its been 4 years.

[DerGut]

Hi @cozzbie, not that I'm aware of but I also haven't followed the project in a while.