search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.33k stars 1.06k forks source link

Add grouping/"cosmetic grouping" to streams #3991

Open DerPhlipsi opened 7 years ago

DerPhlipsi commented 7 years ago

Expected Behavior

I'm just about to create 80+ streams for one device group that is sending logs. This will get very messy on the Stream page, so there should be some kind of grouping function present. Either a simple cosmetic one, see this mock up for example: 2017-07-12_14-01-03 Here you could edit the groups on the right and would be able to assign the streams to the individual groups (multiple selections possible) and decide whether it should be displayed in the default view or not.

Another idea would incorporate a complete grouping system with parent-child-relations. For example the stream NTP is a parent-stream of the child-streams Stratum1, Stratum2, Stratum3 and ServerNTP. The NTP-stream is an aggregation of it's child-streams and acts like any other stream, so if a alert condition that is defined on the NTP-stream is fullfilled by any of the child streams (or a combination of these streams), a notification will be triggered as usual.

I recognize that implementing a parent-child-system is quite time consuming, but I think most Graylog users having bigger production setups with many streams would benefit from such a solution, compensating for the implementation time more than enough. It adds a lot of visibility and also a way to create different levels of abstraction. For example, you could segment your monitoring into multiple levels like:

  1. Global Health Status
  2. Department Health
  3. Team Health
  4. Device Group Health
  5. etc.

Maybe special alerts could further support this idea of parent-child-relations (e.g. If x child-streams have active alerts, trigger an alert --> automatic escalation if a bigger problem occurs.)

Current Behavior

Currently, Graylog does not have any grouping function and only allows for a text filter to search for streams. This can be seen as enough, but will and can get quite messy over time, especially if some well known users use the system 😆

Possible Solution

Idea for the cosmetic variant: Add an attribute to streams that can contain a arbitrary name or id of an group and the frontend will take care of sorting/grouping the streams into the correct tabs.

Context

I'm just about to create 80+ streams for one device group that is sending logs. And this is only for one device group. So I came to this idea.

I somehow have in mind to have read something similar in the past as an issue for graylog, but I'm not sure and am not able to find anything. Please correct me if I'm wrong and there is already an issue/feature request for this.

Your Environment

jalogisch commented 7 years ago

We have this already in out Feature Portal:

https://graylog.ideas.aha.io/ideas/GL2E-I-382 https://graylog.ideas.aha.io/ideas/GL2E-I-382

DerPhlipsi commented 7 years ago

I know, there is also this: https://graylog.ideas.aha.io/ideas/GL2E-I-152. But that idea topic for example references Graylog 1.2, showing its age.

The feature portal doesn't seem to be used so much/frequently, looking at all the uncommented/uncategorized ideas. But this is probably because of you guys being occupied by more important things.