vectordotdev / vector

A high-performance observability data pipeline.
https://vector.dev
Mozilla Public License 2.0
17.41k stars 1.51k forks source link

Document that metric sinks aggregate metrics within a batch #7938

Open bruceg opened 3 years ago

bruceg commented 3 years ago

Most of the metric sinks aggregate metrics within a batch. That is, incremental metrics are added together and absolute metrics are replaced by the later value. As noted by https://github.com/timberio/vector/pull/7846#pullrequestreview-686985284 this is not adequately described in the documentation for those sinks. This applies to the following sinks:

moschroe commented 10 months ago

I was tripped up by this when naively assuming I could import a backlog of metrics from CSV files to influxdb.

My workaround is specifying a batch.max_events: 1, which is terrible for performance, of course.

I my opinion, this aggregation should not occur in the first place, but that is a different discussion, I guess.

StephenWakely commented 10 months ago

Note stackdriver_metrics no longer aggregates within a batch.

curtiseng commented 2 months ago

How long before influxdb_metrics can support turning off auto-aggregation?