inconsistencies and improvements for the SPF settings

[shoulders]

SYSTEM INFORMATION
OS type and version	Ubuntu Linux 22.04.3
Webmin version	2.105
Usermin version	2.005
Virtualmin version	7.8.2
Theme version	21.09.5
Package updates	31 package updates are available

The issue

While setting up my Virtualmin I found the SPF settings to be tricky to use and I will outline the reasons here. Overall the 2 sections should have unified options and be correctly named.

The 2 locations where you find SPF settings. Their issues will be listed below

Edit Template section: DNS domain (Server Template)

Virtualmin --> System Settings --> Server Templates --> 'Default Settings' --> Edit Template section: DNS domain

• Does SPF record cover all senders
  • From what I can tell this is controlling the Failure Qualifier (?all, +all, ~all, -all).
  • This option should be renamed to Failure Qualifier and have all 4 options correctly named as follows:
• TLSA does not have an option to be enabled/disabled here, perhaps it should
  • TLSA is also has an option here and I don't understand why: Virtualmin --> System Settings --> Virtualmin Configuration --> Configuration category: SSL Settings --> Create TLSA DNS records for SSL certificates - The tooltip makes out this enables it for all domains, not just the system domain.
• DMARC policy for emails that fail SPF or DKIM is a radio option where as in DNS Options it is a drop down

DNS Options

Virtualmin --> Server Configuration --> DNS Options --> Action for other senders

• Action for other senders
  • From what I can tell this is controlling the Failure Qualifier (?all, +all, ~all, -all).
  • This option should be renamed to Failure Qualifier and have all 4 options correctly named as follows:
• There is no option to change the DMARC email contacts for Reporting URI for forensic reports and Reporting URI for aggregate reports
• There is no option for Extra DMARC options

The built SPF record has some issues

If you goto Virtualmin --> DNS Records you can see the following

Switches explained

• a = Allow the IP of the website (i.e A record) for sending emails = 31.31.31.31
• mx = Allow the IP of the mail server (i.e. MX record) for sending emails = 31.31.31.31
• a:example.com = Allow the use of the IP for example.com for sending emails - 31.31.31.31
• ip4:31.31.31.31 = Allow the use of the IP 31.31.31.31 for sending emails = 31.31.31.31
• ?all = fail qualifier is neutral

Because you have a and mx you do not then need to define the IP and the domain with the ip4: and a: switches.

You need to keep a and mx because they might have different IPs. The specified resources a: and ip4: should only be used when a user has manually defined a resources or perhaps you have allowed the virtual IP.

Maybe add options to use the A and MX record in the SPF options rather than specifying the IP address.

Resources

• https://simpledmarc.com/blog/spf-failure-understanding-different-types-and-causes/
• https://dmarcly.com/blog/why-spf-authentication-fails-none-neutral-fail-hard-fail-soft-fail-temperror-and-permerror-explained

[jcameron]

So looking at these one at a time ... I'm not sure if using the name "Failure Qualifier" is a good idea, because it's not clear to the average person what this means. That's why I went with "Does SPF record cover all senders?" which I think better explains what it means.

[jcameron]

Also, I don't think +all is every an option we'd want to support since it allows all email ..

[shoulders]

Why you should keep +all

• you already have the all option which is the same as +all. (the Allow option)
• For normal use ?all is almost as bad and you have that one.
• If you are going to use drop down to select the qualifier you need to support all of the options. This particular option cannot cause great harm, just more SPAM. Also sysadmins changing this option know what it is.
• +all is possibly used for testing and is a quick way to make sure emails get delivered.
• Some companies will have external email processors who do notice the small differences between the qualifiers.

Why failure qualifier is a better name

• This is its proper title and it is what it does.
• Action for other senders / Does SPF record cover all senders to me, sounds like email accounts rather than IP addresses that are defined in the SPF record.
• you could use the name SPF Failure Qualifier to make it clear
• Disallow (-all), Discourage (~all) and allow (all) imply that this is what will happen to the emails but it is not correct as it is a test that is applied to the email and then the other systems decided on what they do with the email. i.e.
  • in Spamassassin i you set a SPF_FAIL = 10 and then if you set an auto-delete threshold to 5, the email is then deleted
  • DMARC policy for emails that fail SPF or DKIM - this speaks for itself.
• Changing the option names and making a better tooltip would actually make this clearer because you will be using real industry terms people can google.
• The current value and option names might not translate well into other languages.

Additional

I just added these so I dont forget about them.

< default >

• This should be renamed to None and kept. I removed it from my options above but it is valid.

SPF and DMARC sections should be separated

• These 2 sections can stay on the same page but I would separate them to make the options clearer.

A and MX records should be tested

• Is there a test to make sure the A and MX records and their servers exist because the options don't allow control on this.
• A tick box for the following options:
  • allow domain IP to send emails (A)
  • allow mail server ip to send emails (MX)

SPF issues on Virtualmin.com (for reference) I did an issue a while ago because there was issues with your SPF for the domain sending me the registration verification issue

I am not sure it is fixed

Virtualmin's website SPF records are faulty - Cannot register on forum https://github.com/virtualmin/virtualmin-gpl/issues/560

[iliajie]

Also, I don't think +all is every an option we'd want to support since it allows all email

I think using ~all is more common as it provides a degree of protection while avoiding the risk of legitimate emails being rejected. I'd suggested using ~all instead of ?all (neutral) by default. What do you think?

[shoulders]

I'd suggested using ~all instead of ?all (neutral) by default. What do you think?

• in this world, the only logical option is to set -all
• This is the recommend practice by Microsoft Office 365, cPanel and a few others.
• Why as a default would i let spammers use my domain to send emails.
• Some email services will reject email if ~all is used
• ~all is regarded as a fail in DMARC.

Google is getting strict with accepting emails: https://news.ycombinator.com/item?id=36323137

[iliajie]

Google is getting strict with accepting emails: https://news.ycombinator.com/item?id=36323137

Google historically recommended ~all.

• Why as a default would i let spammers use my domain to send emails.

What if the mail is relayed?

Some email services will reject email if ~all is used

Which are those?

~all is regarded as a fail in DMARC.

DMARC policy decisions are based on SPF or DKIM passing and alignment, not just the SPF policy itself. ~all, i.e. softfail alone does not cause DMARC to fail, as far as I know. If the SPF check passes, and the domain aligns with the "From" domain, then it counts as a pass for DMARC, regardless of the ~all policy.

[jcameron]

One thing I'll do for sure is make the field names for the SPF "all" mode consistent between server templates and the DNS Options page! And expand on the help ...

[shoulders]

~all or -all

Google historically recommended ~all These are example default SPF statements. ~all is definately historic.

• Microsoft: v=spf1 +a +mx +ip4:31.31.31.31 +include:spf.protection.outlook.com -all
• cPanel: v=spf1 +a +mx +ip4:31.31.31.31 -all

What if the mail is relayed?

• Should the headers not be re-written to prevent breaking DKIM?
• an article backing your side, and is a lot more technical than me: https://www.mailhardener.com/blog/why-mailhardener-recommends-spf-softfail-over-fail

Some email services will reject email if ~all is used

I cannot give you any specific examples so might not be widespread, but I have come across it.

~all is regarded as a fail in DMARC.

• https://serverfault.com/questions/942586/does-an-spf-softfail-trigger-dmarc-reject
  • There are no hard and fast rules. Read RFC 7208 section 8
  • The question is if there is a relation between the type of fail resulting from the SPF check. There is not. Only a Pass result will negate the p=reject DMARC policy. I think this means a ~all will still generate a fail as far as DMARC is concerned but this can be changed should you want it

Other

• There should not be a < default > option, but instead one of the following should be preselected (= the same thing but better logic):
  • none
  • pass
  • neutral
  • softfail
  • fail
• Both locations of this setting should use a drop down because this make more sense than radio options.
• First thing a sensible webhoster should do is make sure the SPF template has -all set rather than ~all which from my point of view should be set to -all anyway. The exceptions where people require this change, then they can make it quite easily.
• -all does a better job of reducing spam than ~all

[jcameron]

So is the conclusion that it would make sense to set the default to -all ?

[iliajie]

So is the conclusion that it would make sense to set the default to -all ?

I'd suggest to set the default to ~all. It is a much safer default than -all.