Metrics are important because they help us monitor the processing batch operations and measure performance.
We currently use prometheus in most places (except the new stuff done with a mix of netdata and statsd). It's probably best to stick to the single most widely used metrics in all our system (which is still prometheus). OpenTelemetry is compatible with prometheus and supports also collecting full traces, which are very useful to understanding the performance characteristics of the runs.
Currently in ooni pipeline v5 we were using statsd, but we probably should switch to opentelemetry.
Since OpenTelemetry works by pushing, we don't need to worry about the short running process concerns with exporting them directly via prometheus (that would then have to scraped).
hellais
commented
4 months ago
Metrics are important because they help us monitor the processing batch operations and measure performance.
We currently use prometheus in most places (except the new stuff done with a mix of netdata and statsd). It's probably best to stick to the single most widely used metrics in all our system (which is still prometheus). OpenTelemetry is compatible with prometheus and supports also collecting full traces, which are very useful to understanding the performance characteristics of the runs.
Currently in ooni pipeline v5 we were using statsd, but we probably should switch to opentelemetry.
Since OpenTelemetry works by pushing, we don't need to worry about the short running process concerns with exporting them directly via prometheus (that would then have to scraped).