search

xmidt-org / caduceus

The Xmidt server for delivering events written in Go.
Apache License 2.0
16 stars 19 forks source link

Make sure gauge for outbound queue is 0 on startup #203

Open kristinapathak opened 4 years ago

kristinapathak commented 4 years ago

https://github.com/xmidt-org/caduceus/blob/00ce0db65037d62ca9f45f31f191835e86bcb93b/metrics.go#L150

Instead of making new gauges and other metrics for each outbound sender, we should create these once for caduceus and then pass what is needed to the outbound senders.

Then, we can easily set the gauge to 0 on startup, which hopefully will fix the issue of the gauge getting stuck on a specific queue depth that doesn't look correct:

Screen Shot 2020-05-12 at 1 14 19 PM

Edit: this is unlikely to fix the gauge issue, but would still be a nice fix to make.

kristinapathak commented 4 years ago

Setting the gauge to 0 for each label was done in this pr: https://github.com/xmidt-org/caduceus/pull/205

Fixing the metrics to create counters and metrics once instead of in a variety of different structs can be done at a later time. This will probably be partially or completely done as a part of moving to uber.fx or creating a separate library for the outbound sender.