search

spamhaus / rspamd-dqs

Spamhaus code for RSPAMD Plugin. See https://docs.spamhaustech.com/40-real-world-usage/Rspamd/000-intro.html for instructions
Apache License 2.0
61 stars 11 forks source link

Duplicate Rule Firing #14

Open tjharman opened 3 years ago

tjharman commented 3 years ago

This isn't a bug with the rulset you provide as such, but now I am seeing two firings of XBL.

SPAMHAUS_ZEN (14) [23.129.64.225:from]
MSBL_EBL (7.5) [jchavalii@gmail.com:email]
R_SUSPICIOUS_URL (5) [63.143.41.167]
RBL_SPAMHAUS_XBL (4) [23.129.64.225:from]

The SPAMHAUS_ZEN is the dqs rulset firing, but the RBL_SPAMHAUS_XBL is the built in rspamd rule firing:

Take from /etc/rspamd/modules.conf/rbl.conf

    spamhaus {
      symbol = "SPAMHAUS"; # Augmented by prefixes
      rbl = "zen.spamhaus.org";
      # Check types
      checks = ['received', 'from'];

      symbols_prefixes = {
        received = 'RECEIVED',
        from = 'RBL',
      }
      returncodes {
        SPAMHAUS_SBL = "127.0.0.2";
        SPAMHAUS_CSS = "127.0.0.3";
        SPAMHAUS_XBL = ["127.0.0.4", "127.0.0.5",
            "127.0.0.6", "127.0.0.7"];
        SPAMHAUS_PBL = ["127.0.0.10", "127.0.0.11"];
        SPAMHAUS_DROP = "127.0.0.9";
        SPAMHAUS_BLOCKED_OPENRESOLVER = "127.255.255.254";
        SPAMHAUS_BLOCKED= "127.255.255.255";
      }
    }

I don't know if this is expected behaviour or not? Should some rules not be added somewhere to dqs rules to disable the built in rspamd rules? Or is this left as an exercise to the reader?

Many thanks for the work you do.

ricalfieri commented 3 years ago

Hi, can you try adding in the

/etc/rspamd/local.d/rbl_group.conf

a section like

"RBL_SPAMHAUS_XBL" { weight = 0; }

That should probably take care of this, I probably missed out the rule (I have only RECEIVED_SPAMHAUS_XBL zeroed out)

If it'l solve the problem i'll push a new version

Thanks

tjharman commented 3 years ago

Yup so that seems to have fixed RBL_SPAMHAUS_XBL (it now scores 0) but I'm still seeing other base Spamhaus rules firing. Prime example is the following email:

SPAMHAUS_ZEN (14) [86.34.157.3:from]
FUZZY_DENIED (9.166443) [1:92bfe9835b:0.59:txt]
MSBL_EBL (7.5) [jchavale3ii@gmail.com:email]
ABUSE_SURBL (5) [63.143.41.164:url]
R_SUSPICIOUS_URL (5) [63.143.41.164]
BAYES_SPAM (3.817193) [96.44%]
VIOLATED_DIRECT_SPF (3.5)
RBL_BARRACUDA (3) [86.34.157.3:from]
RBL_ABUSIX (3) [86.34.157.3:from]
HFILTER_HOSTNAME_UNKNOWN (2.5)
FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN (2.5)
RBL_MAILSPIKE_WORST (2) [86.34.157.3:from]
RBL_SPAMHAUS_CSS (2) [86.34.157.3:from]
RBL_WPBL (2) [86.34.157.3:from]
HTML_SHORT_LINK_IMG_1 (2)
RBL_SPAMRATS_SPAM (2) [86.34.157.3:from]
RBL_PSBL (2) [86.34.157.3:from]
RBL_UCEPROTECT1 (2) [86.34.157.3:from]
RBL_TRUNCATE (2) [86.34.157.3:from]
RBL_VIRUSFREE_BOTNET (2) [86.34.157.3:from]
RBL_SENDERSCORE (2) [86.34.157.3:from]
RBL_HOSTKARMA_BLACK (2) [86.34.157.3:from]
RBL_SPAMRATS_DYNA (2) [86.34.157.3:from]
DCC_REJECT (2) [bulk Body=1570 Fuz1=1896 Fuz2=many ]
HTTP_TO_IP (1)
RDNS_NONE (1)
RBL_SPAMRATS_NO_PTR (1) [86.34.157.3:from]
MIME_HTML_ONLY (0.2)
RCVD_NO_TLS_LAST (0.1)
DMARC_POLICY_SOFTFAIL (0.1) [hotmail.com : No valid SPF, No valid DKIM,none]
TO_DN_NONE (0)
FREEMAIL_FROM (0) [hotmail.com]
MX_WHITE (0) []
RCPT_COUNT_ONE (0) [1]
R_SPF_SOFTFAIL (0) [~all:c]
MID_RHS_MATCH_FROM (0)
FROM_HAS_DN (0)
MIME_TRACE (0) [0:~]
FROM_EQ_ENVFROM (0)
ARC_SIGNED (0) [muppetz.com:s=mail:i=1]
RBL_SPAMHAUS_XBL (0) [86.34.157.3:from]
RCVD_COUNT_TWO (0) [2]
ASN (0) [asn:9050, ipnet:86.34.0.0/16, country:RO]
ARC_NA (0)
TO_MATCH_ENVRCPT_ALL (0)
R_DKIM_NA (0)
RBL_SPAMRATS (0) [86.34.157.3:from:127.0.0.43]
FREEMAIL_ENVFROM (0) [hotmail.com]

I see that RBL_SPAMHAUS_XBL now has a 0 score, but RBL_SPAMHAUS_CSS has a score of 2.

Should I just 0 out all the rules in the core rbl.conf file?

returncodes {
        SPAMHAUS_SBL = "127.0.0.2";
        SPAMHAUS_CSS = "127.0.0.3";
        SPAMHAUS_XBL = ["127.0.0.4", "127.0.0.5",
            "127.0.0.6", "127.0.0.7"];
        SPAMHAUS_PBL = ["127.0.0.10", "127.0.0.11"];
        SPAMHAUS_DROP = "127.0.0.9";
        SPAMHAUS_BLOCKED_OPENRESOLVER = "127.255.255.254";
        SPAMHAUS_BLOCKED= "127.255.255.255";
      }

I'd have thought the answer was yes, except for the fact you actually modify RECEIVED_SPAMHAUS_SBL and RECEIVED_SPAMHAUS_CSS in your rbl_group.conf file, and those rules are part of the base install.

I realise at the end of the day it's fully up to me what rules I enable and how I score them, but I'm just trying to understand how the dqs list is supposed to work with the original rspamd Spamhaus RBLs and make sure they're not doubling up.

Thanks again.

slavkoja commented 2 years ago

IMO, it is firing, because the from = false; has no effect against checks enabled by checks option. It was changed sometime before 3.0 was released and here is now (undocumented) exclude_checks option for this.

ricalfieri commented 2 years ago

I don;t really see this as a real "problem". The other rules that fires just give some push to the overall score, but the email should already have been marked as spam by ZEN in any case

slavkoja commented 2 years ago

If it is or is not "real problem" depends on that, if it is false or true positive match, right? As you surely know, false positives can happens and one have to prevent them as it can to not reject legitimate mails. The other problem is, that duplicate rule processing is waste of resources and it doesn't matter, if it is big or small, it is waste.

Anyway, the rule uses bad (improper) option to override built-in RBL rule's checks and thus it simple doesn't do what it have to and this is real problem.

ricalfieri commented 2 years ago

XBL/ZEN has basically a 0 FP rate. I'll revisit this issue in the future as it's low priority but if you have a fix a pull request is welcome