Aggregation Alert based on Lookup Table / Parameter not triggered

[fvjb]

Expected Behavior

The following Alert is expected to trigger: event_code:4625 AND user_name:$sensitive$ AND NOT source:(DEMODC01 OR DEMODC02) It depends on a Parameter, defined in the following Lookup Table:

The Alert does not trigger.

Current Behavior

Configuring the Alert WITH AGGREGATION results in a Preview:

Replaying the Search results in the expected Logs:

This was tested WITH and WITHOUT grouping. Content is attached: content-pack-52c0b672-2e99-4b56-ae99-858fa13b4d2c-1.txt

As soon as the Aggregation is removed, the Alert is triggered as expected.

Context

This came up during a PoC with a customer.

Please watch this video to see the workaround (replacing Parameter with "OR"-statements):

https://github.com/Graylog2/graylog2-server/assets/129726628/a522d649-a329-4206-b66a-2202775ca202

Your Environment

Graylog Version: graylog-enterprise/stable,now 5.1.1-1 amd64 MongoDB Version: mongodb-org/focal,now 6.0.6 amd64 Opensearch Version: opensearch/stable,now 2.6.0 amd64 Browser Version: microsoft-edge-stable/stable,now 113.0.1774.57-1 amd64

[drewmiranda-gl]

Confirming i can reproduce this, and I do see in the logs that the query being sent to the indexer is not resolving the parameter.

Enabling some loggers set to debug level to help:

(originally didn't know exactly which one was the one i wanted so added a few)

org.graylog.plugins.views.search
org.graylog.shaded.opensearch2.org.opensearch
org.graylog.storage.opensearch2

Create Events for Definition if... Filter has results:

event_code:4625 AND source:winser AND user_name:("administrator" OR "Domain Admins" OR "me")

Create Events for Definition if... Aggregation of results reaches a threshold

event_code:4625 AND source:winser AND user_name:$priv_users$

It appears that when using aggregation the parameter is never resolved and sent in the query to the indexer.

[kodjo-anipah]

one of the reason why this can be happening is because /api/views/search requires the parameters that should be used for the search. On the search page, parameter information is passed correctly but this is missing in the event/aggregation view hence the normalization is failing

{
  "id": "optional",
  "queries": [],
  "parameters" : [
    {
      "name": "sensitive",
      "title": "Sensitive",
      "description": "",
      "optional": false,
      "lookup_table": "my-sensitiveusers",
      "key": "fsasdafasdf",
      "type": "lut-parameter-v1",
      "data_type": "any",
      "default_value": "asdfasdfasdf"
    }
  ]
}

[drewmiranda-gl]

Hi @kodjo-anipah quick clarification. I don't fully understand your comment. Is that something a user needs to do or something to fix in the graylog code? Thanks!

[kodjo-anipah]

@drewmiranda-gl sorry for the confusion I continued the investigation and there seems to be inconsistent behaviour so there will be a need for a fix. Will pass this issue over to the search team as they are more familiar with the topic.