[BUG] configtest is non-deterministic

[PAStheLoD]

Prerequisites

• [x] Read about bug reporting in general
• [N/A] Enabled relevant debugging logs
• [N/A] Checked the FAQs about Core files in case of fatal crash
• [N/A] Tried ASAN package and obtained the ASAN report (if possible)
• [x] Checked that your issue isn't already filed
• [x] Checked that there is not already an experimental package or master branch

Describe the bug rspamadm configtest error/warning reporting is non-deterministic, which makes it hard to know if the config is actually correct (or will be eventually correct, or will be eventually incorrect).

Steps to Reproduce

1. install rspamd 3.2

2. 
   cd /etc/rspamd/
   git clone https://github.com/spamhaus/rspamd-dqs.git
   cd rspamd-dqs/
   git checkout 15dbc765ed48ec3d408905ecd57c9626eee1f45d
   cd ../local.d
   ln -s /etc/rspamd/rspamd-dqs/3.x/rbl* .

for i in $(seq 1 10) ; do rspamadm configtest ; done

**Expected behavior**
Deterministic error reporting.

**Actual behavior**

rspamadm configtest

syntax OK

rspamadm configtest

symbol 'URIBL_DROP' has its score defined but there is no corresponding rule registered symbol 'URIBL_SBL' has its score defined but there is no corresponding rule registered symbol 'SPAMHAUS_ZEN_URIBL' has its score defined but there is no corresponding rule registered symbol 'URIBL_PBL' has its score defined but there is no corresponding rule registered symbol 'URIBL_XBL' has its score defined but there is no corresponding rule registered symbol 'URIBL_SBL_CSS' has its score defined but there is no corresponding rule registered syntax OK

### Versions
3.2
ubuntu jammy [amd64, on a Hetzner VM, Xeon Skylake], also tested on ubuntu focal, amd64 [WSL2,  i5-8250U CPU @ 1.60GHz]

[PHPGangsta]

I have the exact same issue. Sometimes it outputs symbol errors, sometimes everything is OK... Without changing anything between both configtests.

michael@testmaschine01:~$ sudo rspamadm configtest
syntax OK
michael@testmaschine01:~$ sudo rspamadm configtest
symbol 'URIBL_DROP' has its score defined but there is no corresponding rule registered
symbol 'URIBL_SBL' has its score defined but there is no corresponding rule registered
symbol 'SPAMHAUS_ZEN_URIBL' has its score defined but there is no corresponding rule registered
symbol 'URIBL_PBL' has its score defined but there is no corresponding rule registered
symbol 'URIBL_XBL' has its score defined but there is no corresponding rule registered
symbol 'URIBL_SBL_CSS' has its score defined but there is no corresponding rule registered
syntax OK
michael@testmaschine01:~$ rspamd --version
Rspamd daemon version 3.2

Ubuntu focal 20.04.5 LTS

[vstakhov]

What is in your local.d/ and override.d/ directories I wonder?

[PAStheLoD]

In my case they are empty apart from the rbl conf. (I'm testing this on a VM that never had rspamd on it before, so everything is as it comes right after apt install + the aforementioned rbl stuff.)

$ ls -alh local.d/
total 8.0K
drwxr-xr-x 2 root root 4.0K Sep 15 18:33 .
drwxr-xr-x 8 root root 4.0K Sep 15 19:38 ..
lrwxrwxrwx 1 root root   35 Sep 15 18:33 rbl.conf -> /etc/rspamd/rspamd-dqs/3.x/rbl.conf
lrwxrwxrwx 1 root root   41 Sep 15 18:33 rbl_group.conf -> /etc/rspamd/rspamd-dqs/3.x/rbl_group.conf
$ ls -alh override.d/
total 8.0K
drwxr-xr-x 2 root root 4.0K Feb 17  2020 .
drwxr-xr-x 8 root root 4.0K Sep 15 19:38 ..

[vstakhov]

And you do have periodic issues with that rbl plugin on configtest... Coincedence? I don't think so. There are likely two rules with the same name and Rspamd randomly selects one of them.

[vstakhov]

I've also checked the dqs files provided by Spamhaus, and I can tell that unfortunately they have the same performance issues I have mentioned several times in the past, so I would not recommend using those in production (e.g. this is a good example how not to write Rspamd rules).

The proper way to disable generic and enable dqs lookups would be to define a new rbl rules in those local files (that do not clash with the existing ones) and disable the generic ones in the same local.d/rbl.conf file.

In any confusion, you can run rspamadm configdump rbl and examine it's output.

[PAStheLoD]

Thanks for looking into it. (I don't plan to use their lua stuff, just the mentioned two files, the RBL ones.)

I've tried to examine the output of configdump to look for clashing symbols/rules/tokens, but by eye I got nowhere and did not have time to write a script yet.

I think at this point the best bet is to try to automatically dissect the config files. Hopefully I'll have some time for this on the weekend.