Regex doesn't work as expected

[drejoe]

Hi,

I'm having some issue with the key/value regex (?:_+)?(?<_KEY_1>[\w.:[]]+)=(?<_VAL_1>.*?(?=(?:\s[\w.:[]]+=|$))) in the TA. Offen the field "Rule Name" (CEF field cs2Label) freaks out and becomes one very long field name (more than 400 chars long). When I test in regex debuggers the regex works fine - but Splunk seems to drop it...

Any idea why?

Cheers //Torben

[tomkopchak]

Can you send over a few sample events (raw data, if possible) where this is happening? Does this happen on only a few rule names, or only one? I'm wondering if there is an issue with a unicode character or something similar that is causing this to happen.

[drejoe]

Sure, how do I sent it to you? (not in this issue threat)

[tomkopchak]

You can send it to me here: https://bigfiles.hurricanedefense.com/filedrop/~gSdYVZ

[tomkopchak]

Thanks for sending that over. What's strange in the example logs is that there's no cs2 field immediately after cs2Label in this sample data. For example, in the same log, preceding, for cs5:

cs5Label=Matched Category cs5=Business / Economy

The way the transforms is set up for this extraction uses the following regex:

[cef_custom_string_extraction]
REGEX = cs[0-9]Label=(.*?)\s+cs[0-9]=(.*?)(?:(?=\s+[A-Za-z0-9]+=)|$)
FORMAT = $1::$2

If these two values aren't next to each other in the log, this will fail.

I notice these are URL Filtering logs, does this happen for every URL filtering log in your firewall? I'm wondering if the format of these events are different for some reason.

[drejoe]

Hi again,

Yes, it seems that all the URL Filtering are having this issue :-/ I'm not an Checkpoint expert - can't say if it's normal or not regarding the "missing" cs2 field when handling URL Filtering.

Cheers //Torben

On Thu, Jan 24, 2019 at 3:46 PM tomkopchak notifications@github.com wrote:

Thanks for sending that over. What's strange in the example logs is that there's no cs2 field immediately after cs2Label in this sample data. For example, in the same log, preceding, for cs5:

cs5Label=Matched Category cs5=Business / Economy

The way the transforms is set up for this extraction uses the following regex:

[cef_custom_string_extraction] REGEX = cs[0-9]Label=(.?)\s+cs[0-9]=(.?)(?:(?=\s+[A-Za-z0-9]+=)|$) FORMAT = $1::$2

If these two values aren't next to each other in the log, this will fail.

I notice these are URL Filtering logs, does this happen for every URL filtering log in your firewall? I'm wondering if the format of these events are different for some reason.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/HurricaneLabs/TA-checkpoint-cef/issues/3#issuecomment-457222153, or mute the thread https://github.com/notifications/unsubscribe-auth/AsX2HQisvTHHankvUrN_J8lr6iMeNAx6ks5vGccugaJpZM4aKtDT .

-- Best regards

Netic A/S

Torben Edelskov Drejø

Senior consultant

Mail: ted@netic.dk

Mobil: +45 2249 1671

Fixed: +45 9635 6180

Se Netics 360° guide til GDPR-compliance http://www.netic.dk/gdpr

[tomkopchak]

Yeah, at least in the sample enviornment I'm looking at, the cs2 field doesn't exist for URL Filtering logs, but I'm not sure if that's how it would behave in every possible configuration. What version of the log exporter and Check Point management server is in use here?

[drejoe]

Hi,

Sorry for the delay - it toke way longer than I expected to get the information from our customer.

Log Exporter R80.10 T41 Security Manager R80.10 T154

Cheers //Torben

On Thu, Jan 24, 2019 at 8:34 PM tomkopchak notifications@github.com wrote:

Yeah, at least in the sample enviornment I'm looking at, the cs2 field doesn't exist for URL Filtering logs, but I'm not sure if that's how it would behave in every possible configuration. What version of the log exporter and Check Point management server is in use here?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/HurricaneLabs/TA-checkpoint-cef/issues/3#issuecomment-457327436, or mute the thread https://github.com/notifications/unsubscribe-auth/AsX2HR7MvkumBWKVKuvt_p5nlPcf5oFiks5vGgrIgaJpZM4aKtDT .

-- Best regards

Netic A/S

Torben Edelskov Drejø

Senior Consultant

Mail: ted@netic.dk

Mobil: +45 2249 1671

Fixed: +45 7777 0871

Se Netics 360° guide til GDPR-compliance http://www.netic.dk/gdpr