search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.43k stars 1.07k forks source link

Correlation of messages #424

Closed GambitK closed 9 years ago

GambitK commented 10 years ago

Right now graylog2 has the streams where you could define rules to match messages, the thing is that you need to know the specifics of what you are looking for.

There's on feature that I think would make graylog2 really powerful, the ability to correlate messages, example:

Alert01: If field "user" has the same value over a period of N time and field "event=300" give me an alert.

Alert02: If field "host" has the same value over a period of N time and field "event=500" give me an alert.

Alert03: If field "host" has the same value and "event=150" is followed by "event=250" over a period of N time alert me.

The idea is that you could do behavior searches and alerts and not have only alerts based on discrete values. I've used this feature extensively in splunk, the alert would send the messages that triggered the rule. I think that's the main missing piece to bridge the gap between them.

jaxxstorm commented 10 years ago

+1

jeffdeville commented 10 years ago

+1 - It seems as if Drools should permit doing this, but I can't figure out if the current Drools support is sufficient for the task. I'm guessing that it is not.

ghost commented 10 years ago

+1 !!!

arabek commented 10 years ago

+1

GambitK commented 10 years ago

A couple of days ago a request came up about having an alert if an ip address repeats itself N times over a period of time asking for a particular resource. I'm currently looking for alternatives for this specific feature to have a tool in parallel to graylog2.

kroepke commented 10 years ago

needs complex event processing support, check what drools supplies already

henrikjohansen commented 9 years ago

See #802 for a possible solution :)

bernd commented 9 years ago

As noted earlier, we have an output plugin for riemann now that can be used with riemann to do these things. https://github.com/Graylog2/graylog2-plugin-output-riemann

I am closing this now because we have lots of similar issue open and we know about this topic. :wink:

Thank you!

pramodanarase commented 8 years ago

What is the status of this issue? which one is the correct issue to monitor ? Can we use Processing Pipelines to find Correlation of messages ? Example - Find the start and end time of any process.

joschi commented 8 years ago

@pramodanarase See https://github.com/Graylog2/graylog2-server/issues/424#issuecomment-76419510