vectordotdev / vector

A high-performance observability data pipeline.
https://vector.dev
Mozilla Public License 2.0
18.2k stars 1.6k forks source link

Sankey Visualizer - Event Flow #21853

Open Aergonus opened 1 day ago

Aergonus commented 1 day ago

A note for the community

Use Cases

Instead of building out a manual flow graph and parsing out buffer/dropped events from internal metrics, I want to be able to generate a graph to view event flows at a glance.

Attempted Solutions

vector graph is great for tracing the topology and internal metrics show how much volume is being processed/dropped but what I really want is an easy way to inflows and outflows for each component.

Proposal

vector sankey events vector sankey event_bytes outputs instantaneous event counts/bytes flows in link format

This would use values like vector_component_sent_events_total, vector_component_received_events_total, vector_buffer_received_events_total, vector_buffer_sent_events_total

References

Simple link format would be ingestible by https://observablehq.com/@mbostock/flow-o-matic and would be 90% of the way to fit https://visactor.io/vchart/guide/tutorial_docs/Chart_Types/Sankey

Version

n/a

pront commented 1 day ago

Hi @Aergonus, this is a good idea. It is unclear at the moment if we can develop this but if you have an implementation plan, please share it. Maybe someone from the community will be motivated to pick this up.