Enrich event message with additional fields

[ihe-nu]

What?

I would love to see the details from my events in extra fields in the message. All events are in the stream "All events". The event-id is stored in the field event_definition_id. I would like to have

• the name of that event
• if the event was grouped by same field the value which triggered the event
• the value of the measurement (count(), card(), ...)

Example: The event looks as following: suspicious ssh logins: 123.123.123.123 - count(ip)=42 event_name: suspicious ssh logins event_grouped_by:123.123.123.123 event_measurement:42

Why?

I wanted to create those extra fields with a grok pattern on the stream. But as the "All Events"-Stream has no inputs, this generates an errormessage:

I have a case where I have to adjust the limits for different users/IPs (=group by values) for the thresholds. As I have quite a few false positives in the beginning I'd like to exclude them and set them to another limit.

Your Environment

• Graylog Version: Graylog 4.0.5 The rest is not relevant I quess.

This issue could also help with #9438

[mpfz0r]

Thanks for the report @ihe-nu

Using pipeline rules or extractors to add fields to events is currently not supported and will probably never will be. However, you can set fields in the event definition itself.

https://docs.graylog.org/en/4.0/pages/alerting/alerting-by-example.html?highlight=template#fields

There are also aggregation fields available to use. Those are unfortunately not documented. Their format is as follows: aggregation_value_<function> or (if a field has been selected): aggregation_value_<function>_<field>

So for example: aggregation_value_count aggregation_value_card_username

With Graylog 4.1 we'll also automatically set fields for the group-by values:

Does that solve your problem?

Refs https://github.com/Graylog2/graylog2-server/pull/10483 https://github.com/Graylog2/graylog2-server/pull/10603

[ihe-nu]

Hi @mpfz0r I think this helps a lot with this particular problem. I am very happy to read about the implementation in Graylog 4.1. In the Overview of the milestones are just 14 open issues left. Do you think it is worth to wait and then have it done automatically? I'm not very motivated to dig through a few dozens events and then update to 4.1 in two weeks.

[mpfz0r]

Hi @ihe-nu I can't make any promises on the 4.1 release date. But expect something by the end of June. When it comes to having access to the group-by values, you'll have no choice as to wait for 4.1. Those were not available before.