search

Kintyre / TA-postfix

Postfix Add-on for Splunk (Compliant with the Mail CIM model)
https://splunkbase.splunk.com/app/3347/
Apache License 2.0
11 stars 3 forks source link

EXTRACT-reject_reason triggers when it shouldn't and collects erroneous results #9

Open barrettnet opened 3 years ago

barrettnet commented 3 years ago

The EXTRACT for reject_reason triggers when it shouldn't (when status!=reject) and collects erroneous results

My suggestion would be to remove EXTRACT-reject_reason and to extract the reject_reason within EXTRACT-status_reject - that way, reject_reason is only extracted when status==reject

Current:

EXTRACT-status_reject = postfix/smtpd\[\d+\]: (?:NOQUEUE|[A-Fa-f0-9]{6,20}): (?\<status\>reject):
EXTRACT-reject_reason = : (?<reject_reason>[^;:]+);

Proposed:

EXTRACT-status_reject = postfix/smtpd\[\d+\]: (?:NOQUEUE|[A-Fa-f0-9]{6,20}): (?\<status\>reject):\s(?<status_code_short>\d+)\s(?\<dsn\>(\d+\\.)+\d+)\s(.+?:\s)?(?<reject_reason>[^;]+);

I've also incorporated extractions for the status_code_short and dsn because they weren't being picked up in these reject events.

The proposed regex works for the data that I have access to but please test against your data.

lowell80 commented 3 years ago

@barrettnet, could you provide a few sample events that demonstrate this? They can be sanitized to strip out any sensitive information, but I realize that we don't have any samples events locally, and those are helpful when checking for regressions in regular expressions over time.

barrettnet commented 3 years ago

My client's policy forbids me from supplying examples but I've found a couple of examples on the internet (https://www.howtoforge.com/community/attachments/mail-log-txt.4247/) which should help:

Jun 6 16:11:00 78comunicacao postfix/smtpd[15648]: NOQUEUE: reject: RCPT from localhost[127.0.0.1]: 550 5.1.1 \contato@78comunicacao.com.br\: Recipient address rejected: User unknown in local recipient table; from=\mailman-bounces@server1.78comunicacao.com.br\ to=\contato@78comunicacao.com.br\ proto=ESMTP helo=

Jun 6 16:17:12 78comunicacao postfix/smtpd[2240]: NOQUEUE: reject: RCPT from pws3.mxtoolbox.com[64.20.227.134]: 454 4.7.1 \test@example.com\: Relay access denied; from=\supertool@mxtoolbox.com\ to=\test@example.com\ proto=ESMTP helo=\<PWS3.mxtoolbox.com>

Jun 6 16:19:04 78comunicacao postfix/smtpd[2295]: NOQUEUE: reject: RCPT from unknown[187.64.38.88]: 504 5.5.2 \<FLY>: Helo command rejected: need fully-qualified hostname; from=\contato@78comunicacao.com.br\ to=\contato@78comunicacao.com.br\ proto=ESMTP helo=\<FLY>

Jun 6 19:25:03 server1 postfix/submission/smtpd[2431]: NOQUEUE: reject: RCPT from 189-71-122-217.user.veloxzone.com.br[189.71.122.217]: 554 5.7.1 \<189-71-122-217.user.veloxzone.com.br[189.71.122.217]>: Client host rejected: Access denied; from=\douglas@78comunicacao.com.br\ to=\douglas@78comunicacao.com.br\ proto=ESMTP helo=\<UserPC>

Chris

lowell80 commented 3 years ago

Thanks again for taking the time to provide feedback and pull up some samples. I really appreciate it!

Unfortunately, I think somethings happening to the regex formatting when they are posted in the github issue. On the last one I could just edit your initial description text and get the original from there, but that doesn't seem to be working this time. Could you try reposting your regex in triple backticks?

For example, you listed this as the current version:

EXTRACT-status_reject = postfix/smtpd\[\d+\]: (?:NOQUEUE|[A-Fa-f0-9]{6,20}): (?\<status\>reject):

But what I have in the repo as of b03478bb is following:

EXTRACT-status_reject = postfix/smtpd\[\d+\]: (?:NOQUEUE|[A-Fa-f0-9]{6,20}): (?<status>reject):

Maybe the only difference is the \ around the <>?, or possibly you have an older version installed? I fiddled with it for a few mins and it wasn't matching the sample events you provided. (Or at least the one I grabbed)

Alternately, you could provide the updated regexes as a pull request and avoid weird formatting issues. You can do the whole process in the browser if you'd like. GitHub makes the whole thing pretty easy: https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request

barrettnet commented 3 years ago

Yes, those extra slashes preceeding the < and > for the status and dsn extractions are likely the issue. I was flipping between the Write and Preview tabs to try and get it formatted right but I may have messed up. I'm at another client at the moment but I'll see if I can get something sorted when I'm back in with my main client tomorrow.