vectordotdev / vector

A high-performance observability data pipeline.
https://vector.dev
Mozilla Public License 2.0
17.48k stars 1.53k forks source link

Document what the difference between stateless-aggregator and aggregator roles is for the Helm chart #13222

Open jszwedko opened 2 years ago

jszwedko commented 2 years ago

I couldn't find any documentation about this aside from the minimal mention in the Helm chart configuration option docs, which don't provide detail about what the difference is between the two aggregator roles and when to use which.

As a user, I'd expect to see this documented in the following places:

For those that stumble upon this issue, the difference is that the stateless aggregator doesn't require any persistent volume claims. The stateless aggregator is preferred if your Vector configuration doesn't require a disk: that is that it doesn't use disk buffers or any sources that require checkpointing (like the file and journald sources).

spencergilbert commented 2 years ago

Last time @binarylogic and I had discussed this he'd preferred to either remove it or include it in a much more hidden advanced configuration sort of section. I'm curious if that's changed.

jszwedko commented 2 years ago

@spencergilbert I'm open to that. Which would we remove? Some users seem to want to avoid using persistent volumes if they can, but we do require them for certain use-cases.

Until we remove it, if we do, I think just documenting it would help a lot.

spencergilbert commented 2 years ago

I'll see if I can find the previous conversation (I assume it's been killed by Slack retention) - but @binarylogic's position was prioritizing the StatefulSet.