Questionable use of Public Suffix List (PSL) in helo.checks module

After enabling the valid_hostname check in the helo.checks module, we found that our Haraka hosts could no longer send mails to each other, and we tracked this down to being an issue related to the PSL. I'm not reporting this as a bug because I believe the code works as intended, but I am rather questioning the idea behind that code because I believe there's others suffering from this, too.

Background:

We do web and mail hosting for customers. Our host names follow the scheme <hostname>.uberspace.de, and when a user registers an account, he can access it through https://<username>.<hostname>.uberspace.de before even adding a custom domain. So far, so good.

Mostly to prevent "supercookies" - one of the main reasons for the existence of the PSL! - we decided to put *.uberspace.de onto that list, so that browsers would not allow e.g. alice.fairydust.uberspace.de to set or steal cookies of uberspace.de or any other user's site under a subdomain of ours. In the PSL terminology, Alice would "register" the "domain" named "alice" under the "TLD" named "fairydust.uberspace.de", matched by the wildcard entry on the PSL.

Back to Haraka. plugins/helo.checks.js does the following in the exports.valid_hostname function:

    // this will fail if TLD is invalid or hostname is a public suffix
    if (!tlds.get_organizational_domain(helo)) {

Here's the problem: Our hostnames follow the naming scheme *.uberspace.de, so the host fairydust.uberspace.de would greet other mailservers by sending EHLO fairydust.uberspace.de.

If Haraka's valid_hostname check is enabled, it rejects all of our hosts. Here's an example SMTP session:

$ nc stardust.uberspace.de 25
220 stardust.uberspace.de ESMTP Haraka/3.0.2 ready
EHLO fairydust.uberspace.de
550 HELO host name invalid

Now that's a bummer because fairydust.uberspace.de is a perfectly valid hostname, with A/AAAA records and matching PTR records for its IPs. It is denied because tlds.get_organizational_domain(helo) does not return an "organizational domain" which is based on the PSL entry in etc/public-suffix-list of the haraka-tld module. I tried to manually remove the entry from the local copy of the PSL, and the problem is immediately solved.

I am questioning if it's a good idea to use the PSL here at all. The PSL was made to be used by browsers. Neither https://publicsuffix.org/ nor https://en.wikipedia.org/wiki/Public_Suffix_List list any indication that it was made for the use of mailservers, and I am not aware of any other mailserver using it to check if a connecting hostname is valid.

It's not a problem we can solve on our side: Removing *.uberspace.de from the PSL is out of reach as it would expose our users to supercookies, and while we can control our config, we cannot control the configs of other Haraka instances in the world - while it affects our ability to send mails to any Haraka servers having the valid_hostname check enabled, despite our hostnames being perfectly valid. For sure we can hack exceptions into our own config, but that wouldn't fix the problem for others.

We're also not the only ones affected: Other - not so small - hosting providers did the same as we did, like Amazon, OVH or Salesforce (code.com), having these entries on the public suffix list:

*.compute.amazonaws.com
*.compute-1.amazonaws.com
*.compute.amazonaws.com.cn
*.elb.amazonaws.com
*.elb.amazonaws.com.cn

*.webpaas.ovh.net
*.hosting.ovh.net

*.builder.code.com
*.dev-builder.code.com
*.stg-builder.code.com

All hosts matching these wildcards would be unable to deliver mails to any valid_hostname-enabled Haraka server, too.

Final words: https://publicsuffix.org/learn/ lists what the PSL is commonly used for, which is basically a) browsers and b) libraries. Under the "Other" section, two of the three mentioned other use cases are explicitly labelled with a warning: "third party limits are not something the PSL was designed for".

So I would like to humbly ask: Is there room for considering to stop using the PSL for Haraka checks?

If Haraka's valid_hostname check is enabled, it rejects all of our hosts. Here's an example SMTP session:

That's not quite accurate. This is connecting to a Haraka server with the default configuration of helo.checks:

$ nc wildduck.tnpi.biz 25
220 mail.tnpi.biz ESMTP Haraka/3.0.3 ready
EHLO fairydust.uberspace.de
250-mail.tnpi.biz Hello cali.tnpi.net [2605:7900:20:a::6]Haraka is at your service.
250-PIPELINING
250-8BITMIME
250-SMTPUTF8
250 SIZE 26214400
quit
221 mail.tnpi.biz closing connection. Have a jolly good day.

The default configuration of helo.checks is to check whether the hostname is valid, but the default does not reject based on it.

[NOTICE] [core] connect ip=2605:7900:20:a::6 port=23411 local_ip=2605:ae00:329::c local_port=25
[INFO] [fcrdns] ip=2605:7900:20:a::6  rdns="cali.tnpi.net" rdns_len=1 fcrdns="cali.tnpi.net" fcrdns_len=1 other_ips_len=0 invalid_tlds=0 generic_rdns=true
[INFO] [helo.checks] helo_host: fairydust.uberspace.de, pass:match_re, bare_ip, dynamic, big_co(not), host_mismatch, fail:valid_hostname, rdns_match, forward_dns(invalid_hostname)
[NOTICE] [core] disconnect ip=2605:7900:20:a::6 rdns=cali.tnpi.net helo=fairydust.uberspace.de relay=N early=N esmtp=Y tls=N pipe=N errors=0 txns=0 rcpts=0/0/0 msgs=0/0/0 bytes=0 lr="" time=21.892
[WARN] [core] data after disconnect from 2605:7900:20:a::6

Arguably, a number of the HELO/EHLO checks aren't suitable for rejecting mail connections. Which is why there's a separate [reject] option that defaults to being disabled. The easy solution here is for you not to set [reject]valid_hostname=true.

haraka / Haraka

Questionable use of Public Suffix List (PSL) in helo.checks module #3209