NOT (!) Operator functioning incorrectly in pipeline rule

[dubsout]

Expected Behavior

When writing pipeline rule filtering by IP in when clause, the NOT operator should filter messages based on IP range specified.

Current Behavior

When writing rules with NOT operator, it ignores the rule and matches anyway.

Possible Solution

Steps to Reproduce (for bugs)

Create a stream with specific messages you are interested coming in. Connect this stream to a pipleline. Add a stage that checks against a specific IP range like below (have tried with || operator as well)

rule "checks if address is NOT part of guest network"
when
  ! cidr_match("10.101.130.0/23", to_ip($message.IPAddress)) && 
  ! cidr_match("10.101.134.0/23", to_ip($message.IPAddress))
then
end

On the second stage, route to a clean stream.

rule "routes to DHCP Guest stream"
when
  true
then
  route_to_stream(name: "Stream Name");
end

You will find the correct IP addresses are routed to this stream along with the ones we do not want above.

Context

Instead of having to write out hundreds of subnets to match, I would like to not match against a couple to keep the rules concise.

I can successfully filter messages when specifying the subnets I want to match, but am unsuccessful with the syntax above.

Your Environment

• Graylog Version: 2.1.1+01d50e5
• Elasticsearch Version: 2.3.5
• MongoDB Version: 3.2.9
• Operating System: Ubuntu 16.04 LTS
• Browser version: Chrome

[hc4]

Use != true as workaround. functions returning bool works strange in rule conditions by now.

[kroepke]

@hc4 What do you mean by that? Boolean valued functions are fine to use in expressions (there are tests for exactly that).

[hc4]

I've created issue about that https://github.com/Graylog2/graylog-plugin-pipeline-processor/issues/85

[kroepke]

@dubsout I have replicated that setup an it works fine for me. What you describe should work.

I'll attach a few screenshots to illustrate:

Pipeline setup (attached to the "default stream", which means all incoming messages):

Rule with cidr_match:

Stream routing rule:

Messages: I simplified the message a bit, for ease of sending (using a raw input and the message field).

The stream has no legacy stream rules attached to it.

[kroepke]

@hc4 Ah ok, I thought it was broken in general. As I've alluded to in that issue, the interpreter can't figure out the type of the property access (.matches) right now, it is not a function call. The interpreter current only treats function calls specially. We'll address that in the next version. Thanks!

[hc4]

Just missed .matches access :)

[dubsout]

@kroepke Sorry for making you go digging into this, I just realized my mistake in logic. I broke up the subnets into multiple rules and this will naturally let messages pass through since it will only not match in one or the other, not both.

[dubsout]

@hc4 thanks for pointing out the above - even though that wasn't the exact issue, i found my mistake because of it

[kroepke]

@dubsout No problem :)

If you have a suggestion in how to make it more obvious to see what's going on, please report an issue/feature request at https://github.com/Graylog2/graylog-plugin-pipeline-processor/issues

Thanks!