search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.37k stars 1.06k forks source link

Rules engine for messages #893

Closed henrikjohansen closed 8 years ago

henrikjohansen commented 9 years ago

At the moment you can't say that if a message matches this regex, contains this 'thing', comes from this source, etc then run these 25 extractors.

If all you need are a few log formats and you can afford to use different inputs for each of them chances are that you're probably going to be just fine. If you have many different formats and perhaps even a few appliances, etc where you cannot change the syslog port you're going to feel the pain.

You can easily find yourself in a situation where a single input has more than 100+ individual extractors or a situation where you run 50+ inputs (a separate input per log type) which make maintenance a real nightmare.

This also makes it much easier to enforce a strict nomenclature (like prefixing all fields matching a rule) to avoid unintentional field name clashes with the dynamic mappings in ES.

This concept could also be useful for radio nodes - i.e. deciding which queue a message should be routed to ...

kroepke commented 9 years ago

relates to #895

mikkolehtisalo commented 8 years ago

How much of this is not resolved with the processing pipelines feature of 2.x?

henrikjohansen commented 8 years ago

@mikkolehtisalo Most, if not all of it.

kroepke commented 8 years ago

Please file reports about missing features over at https://github.com/Graylog2/graylog-plugin-pipeline-processor/issues :)

rfdrake commented 8 years ago

This seems to be a similar idea to #765

bernd commented 8 years ago

Closing this. Please report missing pipeline functions over at https://github.com/Graylog2/graylog-plugin-pipeline-processor/issues. Thank you!