search

brightcove / hot-shots

Node.js client for statsd, DogStatsD, and Telegraf
MIT License
527 stars 135 forks source link

How to debug data not arriving to DD? #257

Open GeoffreyPlitt opened 1 year ago

GeoffreyPlitt commented 1 year ago

I am sending successful hotshots "increment" calls-- the callback returns a null error, and positive number of bytes.

However, the data doesn't seem to get to DataDog.

In fact, when I run locally, without even having any StatsD server running, the calls still look successful.

Also, I turned on debug logging (using env var "DEBUG=* node test.js"...) and I don't see any logs generated by this library.

I don't understand. How does this library know if data gets to a stats server or not? How do I debug this? Is there a log somewhere that should show me datapoints enqueued and not delivered?

jakub-g commented 1 year ago

@GeoffreyPlitt did you manage to figure things out?

I'm also debugging an issue where data silently stopped being sent, and I realized that by default, this lib uses UDP, which per definition is not reliable. But it can be configured to use a different transport.

https://github.com/brightcove/hot-shots/blob/b4422446831b47873637b43ff6d111b1cef1b5d5/lib/transport.js#L152

GeoffreyPlitt commented 1 year ago

I have not figured this out. DataDog sent me down a rabbit hole of running local agents and sending flares that didn't work out, and when I tried to switch to a fresh account (our main account was too noisy), they told me non-paid accounts aren't allowed to send custom metrics.