monogon-dev / NetMeta

NetMeta is a scalable network observability toolkit optimized for performance.
https://netmeta.demo.monogon.dev/
Apache License 2.0
133 stars 8 forks source link

Allow static sampling-rate overwrite #102

Closed vidister closed 1 year ago

vidister commented 1 year ago

Some routers (for example Cisco IOS-XE) don't include a sampling rate field in their Netflow/IPFix export. This leads to Grafana only displaying zero values. (Default sampling rate is 0, so bytes/packets get multiplied by 0)

I want to propose we add an exporter-map to the NetMeta configuration file to configure the sampling rate per source IP. In the future this map could be useful to add an alias to the exporters, so we can have human friendly names in the dashboards.

vidister commented 1 year ago

I see two ways to implement this:

I'm not sure which fits better to the plans of Netmeta.

leoluk commented 1 year ago

Look up the sampling rate on query-time

How could it be queried?

vidister commented 1 year ago

A materialized column should work for this. I don't know all the fancy bells and whistles of Clickhouse that well, so maybe there's a prettier way that I don't see.

I feel like the boilerplate to query in a way that is compatible with the various quirks of different setups is getting a bit much. Maybe it makes sense to settle on materialized views to abstract away this kind of complexity and provide a stable-ish interface to the users?

leoluk commented 1 year ago

What I mean is - how would you query it?

leoluk commented 1 year ago

Lookup map by exporter sounds fine to me (same as we already do for the interfaces)