regex($message.message) does not work

[jkroepke]

Problem description

let field = regex(".*(\d+).*" ,to_string($message.message), ["number"]);

Steps to reproduce the problem

1. Create a pipeline with a rule:

let field = regex(".*(\d+).*" ,to_string($message.message), ["number"]);
set_field("number", field.number);

The field number will not appear in messages.

Create another pipeline with a rule:

set_field("regex_message", to_string($message.message));
let field = regex(".*(\d+).*" , to_string($message.regex_message), ["number"]);
set_field("number", field.number);

The field number will be appear with some numbers..

Environment

• Graylog Version: 2.0.3
• Pipeline Processor plugin version: graylog-plugin-pipeline-processor-1.0.0-beta.5.jar (shipped via graylog rpm)?
• Elasticsearch Version: 2.3.3
• MongoDB Version: 3.0.12
• Operating System: Mac OSX 10.11
• Browser version: Chrome 52

[jalogisch]

@jkroepke when you check your regex with some regex tester you notice that you need to modify your regex to make it work.

rule "whatever"
   when
      true
   then
    let field = regex(".*?(\\d+).*?" ,to_string($message.message), ["number"]);
    set_field("number", field.number);

    set_field("regex_message", to_string($message.message));
    let field2 = regex(".*?(\\d+).*?" , to_string($message.regex_message), ["number"]);
    set_field("number2", field2.number);
   end

or in short: first, use \\ and then make the .* non-greedy with .*?

Please check it on your end and reopen this issue if still present.

[jkroepke]

@jalogisch Sorry for the issue in my example. We used a long regex with double backslashs.. The error still exists.

[jkroepke]

Unable to reopen this issue, when a member close it.

[jalogisch]

@jkroepke could you please give a nonworking example.

include the rule and example message.

thank you

[jkroepke]

Okay.

Message:

SNMP_TRAP_LINK_DOWN: ifIndex 697, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-3/0/2

Non working rule:

let field = regex(".*(^[A-Z_]+):\\s+\\w+\\s([^,]+),\\s+\\w+\\s([^,]+),\\s+\\w+\\s+([^,]+),\\s+\\w+\\s+([\\w\\d-\\/]+)",to_string($message.message), ["action", "ifindex", "ifadminstatus", "ifoperstatus", "ifname"]);

Working rule:

set_field("regex_message", to_string($message.message));
let field = regex(".*(^[A-Z_]+):\\s+\\w+\\s([^,]+),\\s+\\w+\\s([^,]+),\\s+\\w+\\s+([^,]+),\\s+\\w+\\s+([\\w\\d-\\/]+)",to_string($message.regex_message), ["action", "ifindex", "ifadminstatus", "ifoperstatus", "ifname"]);
remove_field("regex_message");

Quicklink to your regex tester: http://fiddle.re/uanz5a

[kroepke]

At least with the most recent 2.1.0 snapshot I cannot reproduce this, I just don't have a 2.0.3 install where I quickly check that, @jalogisch do you have one?

My rule is:

rule "test-82"
when true
then
let field = regex(".*(^[A-Z_]+):\\s+\\w+\\s([^,]+),\\s+\\w+\\s([^,]+),\\s+\\w+\\s+([^,]+),\\s+\\w+\\s+([\\w\\d-\\/]+)",to_string($message.message), ["action", "ifindex", "ifadminstatus", "ifoperstatus", "ifname"]);
set_fields(field);
end

[jalogisch]

@kroepke if I use your rule in a fresh 2.0.3 OVA and the message given by @jkroepke the result looks like it should (as far as I had understood the initial request).

• Graylog 2.0.3 (f07c170) on graylog
• Pipeline Processor Plugin 1.0.0-beta.5

[kroepke]

That looks good to me @jalogisch @jkroepke Could you share the entire rule, please? There must be something missing :)

Thanks

[jkroepke]

After retest it with $message.message, looks like working fine now...

Sorry about the confusion and waste your time :>