search

skupperproject / skupper

Skupper is an implementation of a Virtual Application Network, enabling rich hybrid cloud communication.
http://skupper.io
Apache License 2.0
595 stars 74 forks source link

Collector logging #1783

Open c-kruse opened 1 week ago

c-kruse commented 1 week ago
c-kruse commented 1 week ago

@ajssmith wanted to get your take on this approach before cleaning it up a bit. I'm still on the fence about the "flow-logging-profile" flag.

Background: In skupper v1 the volume of logs coming from a collector handling a lot of vanflow records could be a problem. In v2 I dropped flow logging to avoid the issue. For the most part I have not missed them - I have been leaning more on some of the new internal metrics the collector publishes, but there's been a time or two I wish I had them.

Starting out on the problem, I went with a fully user configurable per-record-type-regex-matching sampling but found that was too much to configure as a user. I've since moved to offering "profiles" with different sampling strategies set at somewhat arbitrary levels. Thoughts on this approach? Other suggestions?

EDIT Went forward with cleaning out some of the original cruft. Considering this fully ready for review!