search

DataDog / java-dogstatsd-client

Java statsd client library
MIT License
176 stars 102 forks source link

feat: Support Dogstatsd 1.1 protocol for distributions #216

Closed blemale closed 9 months ago

blemale commented 1 year ago

There is a new experimental Dogstatsd protocol (1.1) which supports sending multiple values in a single datagram.

This is supported since agent 7.25.0/6.25.0.

This evolution in the protocol allows clients to buffer histogram and distribution values and send them in fewer payload to the agent (providing a behavior close to client-side aggregation for those types).

This commit add support for this new protocol for distribution.

Fix #215

blemale commented 1 year ago

@vickenty do you have an idea why CircleCI is complaining as I have not touch anything related to it?

CircleCI Pipeline — Could not find a usable config.yml, you may have revoked the CircleCI OAuth app.

vickenty commented 1 year ago

Hi @blemale , thank you for opening the PR, but I don't think this is the right way to do this: multi-valued payloads are not a feature that we expose directly to the users, but rather than an optimization that can be toggled on and off (via "client side aggregation").

For example, please see how datadog-go implements this: clients provide one sample at a time, which are then aggregated internally and sent as a multi-valued message. Serialization code should also take care to split values over several messages to avoid too big payloads.

blemale commented 1 year ago

👋 Hi @vickenty ! Thanks for the insights :)

Sadly, on the code base i'm working on, i.e DD logs-backend, we are not using the aggregation layer, i.e .enableAggregation(false), as we use dropwizard-metrics that already does the aggregation and only reports point to dogstatsd through this library every 10 or 20s.

However every time we report a distribution we report 1 000 values, currently using recordDistributionValue, which create burst of points on the dogstatsd agent and so this proposal to let client code submit several points in one datagram.

I understand that this proposal goes a bit against the design of this library but dropwizard-metrics has been the go to library for metrics java for a long time and it already does the aggregation. So having a way to integrate efficiently dropwizard-metrics and java-dogstatsd-client would be really appreciated.