bug report: Rspamd can not find the key to sign.

ckvist72 commented 1 year ago

📝 Preliminary Checks

[X] I tried searching for an existing issue and followed the debugging docs advice, but still need assistance.

👀 What Happened?

Rspamd can not find the key to sign.
cannot load dkim key /var/lib/rspamd/dkim/mydomain.dk.dkim.key

👟 Reproduction Steps

Made teh key with the helper-script.

🐋 DMS Version

Latest

💻 Operating System and Architecture

Ubuntu 22.04

⚙️ Container configuration files

No response

📜 Relevant log output

No response

Improvements to this form?

No response

polarathene commented 1 year ago

Did you follow these instructions from our DKIM (Rspamd) docs?:

Opt-in via ENABLE_RSPAMD=1 (_and disable the default OpenDKIM: ENABLE_OPENDKIM=0_).
docker exec -ti <CONTAINER NAME> setup config dkim
The script will persist the keys in /tmp/docker-mailserver/rspamd/dkim/. Hence, if you are already using the default volume mounts, the keys are persisted in a volume.
In case you have not already provided a default DKIM signing configuration, the script will create one and write it to /etc/rspamd/override.d/dkim_signing.conf. If this file already exist, it will not be overwritten. When you're already using the rspamd/override.d/ directory, the file is created inside your volume and therefore persisted correctly. If you are not using rspamd/override.d/, you will need to persist the file yourself (otherwise it is lost on container restart).

You did not provide any config, or information about your DKIM files generated still existing after you encountered this error:

Rspamd can not find the key to sign. cannot load dkim key /var/lib/rspamd/dkim/mydomain.dk.dkim.key

That path doesn't match what the docs suggest it should, which probably means you've not configured DMS properly like the docs describe. Please review that again. Look for the generated DKIM files, and make sure that path is the one used in the DKIM config used by rspamd.

ckvist72 commented 1 year ago

I tested a little. And i also found the problem. Pretty simple... Everything is made correctly, but when i make the DKIM files with the command for this, there is not added an entry in dms/config/rspamd/dkim_signing.conf.

When i add this extra domain-entry manuel, everything works fine.

I have testet on a brand new docker container. Same issue. - And same solution. :-)

ckvist72 commented 1 year ago

Sorry, forgot to write... When i make DKIM key for the FIRST domain, the domain entry is made in the override file. But when i make a key for the nest domain, the domain entry is NOT added to the override file.

polarathene commented 1 year ago

When i make DKIM key for the FIRST domain, the domain entry is made in the override file. But when i make a key for the nest domain, the domain entry is NOT added to the override file.

Yes.. the docs I quoted above explained this. Here's a screenshot:

docs rspamd

jessec commented 1 year ago

If I create 2048 keys my emails won't get signed when I change to 1024 everything works.

polarathene commented 1 year ago

@jessec this is probably user error on your end, or an issue with your DNS service.

1024 keys can fit into a normal DNS TXT record response, but 2048 or higher needs special formatting to split the response. Most DNS providers should support that, but syntax may vary.

You can learn about this from our DKIM DNS section, there is a collapsed section that you can click on to expand, and it will explain this:

That provides a link for when some DNS services don't support the longer keys as well (like AWS Route 53).

jessec commented 1 year ago

I thought about this, but even if my DNS provider has problems with 2048 keys. DMS should still be able to sign the outgoing emails right? I would expect a signed email but it would not validate because of an incorrect DNS entry. BTW is there a strong reason why I should not use a 1024 key?

This is my dkim_signing config. And even after I added the domains manually it will only sign the first (domain.pro):

# documentation: https://rspamd.com/doc/modules/dkim_signing.html

enabled = true;

sign_authenticated = true;
sign_local = true;

use_domain = "header";
use_redis = false; # don't change unless Redis also provides the DKIM keys
use_esld = true;

check_pubkey = true; # you wan't to use this in the beginning

domain {
    domain.pro {
        path = "/tmp/docker-mailserver/rspamd/dkim/rsa-1024-mail-domain.pro.private.txt";
        selector = "mail";
    }
    domain.online {
        path = "/tmp/docker-mailserver/rspamd/dkim/rsa-1024-mail-domain.online.private.txt";
        selector = "mail";
    }
}

I tried this too:

domain {
    domain.pro {
        selectors [
            {
                        path = "/tmp/docker-mailserver/rspamd/dkim/rsa-1024-mail-domain.pro.private.txt";
                        selector = "mail";
            }
        ]
    }
    domain.online {
        selectors [
            {
                        path = "/tmp/docker-mailserver/rspamd/dkim/rsa-1024-mail-domain.online.private.txt";
                        selector = "mail";
            }
        ]
    }
}

jessec commented 1 year ago

Sorry, forgot to write... When i make DKIM key for the FIRST domain, the domain entry is made in the override file. But when i make a key for the nest domain, the domain entry is NOT added to the override file.

Is it possible to see your config file to check if it is the same as mine see above.

jessec commented 1 year ago

OK problem solved the config in the docker volume:

docker-data/dms/config/rspamd/override.d/dkim_signing.conf

was not the same as the config inside the container:

/etc/rspamd/override.d/dkim_signing.conf

After I copied the updated config with the 2 domains over the old config in the etc folder inside the container and did a restart everything worked.

update:

I did everything again with a new server and now the "docker volume config" docker-data/dms/config/rspamd/override.d/dkim_signing.conf was picked up so I think something went wrong on my side.

polarathene commented 1 year ago

I thought about this, but even if my DNS provider has problems with 2048 keys. DMS should still be able to sign the outgoing emails right? I would expect a signed email but it would not validate because of an incorrect DNS entry

You should be able to sign offline yeah, but I'm not familiar with what rspamd is doing. I do recall the docs describing some features that do look up the DNS records even for signing I think (but configurable).

I don't work on the integration with DMS, so if anything like that is enabled it could possibly be related to that. EDIT: Seems you identified it as a sync issue with the config the container had access to 👍

BTW is there a strong reason why I should not use a 1024 key?

It should be ok 👍

It's not as risky as with keys elsewhere like for SSL / TLS. DKIM keys are only to validate / authenticate trust in the delivered message contents not being tampered. Not as valuable to attack.

I don't think there's been any public attacks above 800-bit RSA keys, attacking 1024-bit RSA keys should still be quite expensive today that it's not likely to be a concern for most (all?) DMS users. Especially if you rotate (replace) the keys a few times a year (reduced window for an attacker to leverage).

2048-bit RSA keys are still quite strong, and IIRC would require close to the energy requirements to boil all the oceans in the world 😅 (quantum computing supposedly can do it quicker with enough proper qubits, but I'm not sure if cost to compute for hardware + power was less)

Anything higher is usually excessive imo and a result of security paranoia from those that understand the math less (4096-bit RSA is not double the difficulty of 2048-bit RSA for example, and wastes computation vs adopting ECC/Ed25519).

github-actions[bot] commented 1 year ago

This issue has become stale because it has been open for 20 days without activity. This issue will be closed in 10 days automatically unless:

a maintainer removes the meta/stale label or adds the stale-bot/ignore label
new activity occurs, such as a new comment

simonwiles commented 1 year ago

I'm having the same trouble (dkim_module_load_key_format: cannot load dkim key) but I seem to have everything set up correctly.

brand new clean environment, ghcr.io/docker-mailserver/docker-mailserver:12.1.0
bring up the container, create the first user, run docker exec -it setup config dkim domain sub.example.org

keys are created, and rspamd/override.d/dkim_signing.conf is created:

enabled = true;

sign_authenticated = true;
sign_local = true;

use_domain = "header";
use_redis = false;   # don't change unless Redis also provides the DKIM keys
use_esld = true;

check_pubkey = true; # you wan't to use this in the beginning

domain {
    sub.example.org {
        path = "/tmp/docker-mailserver/rspamd/dkim/rsa-2048-mail-sub.example.org.private.txt";
        selector = "mail";
    }
}

However, my messages are not signed, and /var/log/supervisor/rspamd.log contains

dkim_module_load_key_format: cannot load dkim key /var/lib/rspamd/dkim/example.org.dkim.key: cannot stat key file: '/var/lib/rspamd/dkim/example.org.dkim.key' No such file or directory

My override.d/dkim_signing.conf is correctly mapped into the container at /etc/rspamd/override.d/, and the correct config shows up in the output of rspamadm configdump (and there is no mention there of /var/lib/rspamd/dkim/example.org.dkim.key, nor does that path appear anywhere else I can find in the config).

DNS is all good and opendkim-testkey is happy, but that seems irrelevant since the messages aren't being signed in the first place.

Does anyone know what's happening here? Is the subdomain relevant? Very grateful for any pointers! Thanks!

polarathene commented 1 year ago

I think in this case, you're trying to send mail with a @example.org address, which should work if you did docker exec -it setup config dkim domain example.org.

Perhaps you had a misunderstanding, and configured for your mail-server FQDN (mail.example.org), not the domain-part of your mail accounts that you want DKIM signatures for?

@georglauterbach might be able to weigh on that as he manages the Rspamd support.
You might be able to check with :edge in case it was addressed already (I know some DKIM + rspamd fixes landed since).

By default if you have mail.example.com assigned as the hostname for DMS, and then you want to manage email accounts like hello@example.com; the setup config dkim command should implicitly create a DKM key for example.com with a DKIM selector mail (when no domain option is provided). This has been clarified in the setup config dkim help output and our DKIM docs page in :edge, it's a default that would work for most users expectations.

Possibly misconfigured

This isn't an area I know too well beyond what our scripts do. I would need to hit the search engines to confirm, but our current docs and support seem to imply you create the DKIM keys for each managed mail domain (example.com) and not the mail server itself that is doing the signing (mail.example.com / mail.another-example.com).

Although I get the impression that it may not matter, and only one DKIM private key is actually needed for DMS to sign with for all mail domains it manages (while each domain has DNS records for DKIM with the selector to support verification with the DKIM public key).

Likely cause: If that's the case, then on the DMS end the configs are only to say "that domain is configured for DKIM and here's the DKIM private key to sign with". So in your case sub.example.org may expect that you're wanting to sign for mail with a sender address of @sub.example.org not @example.org, while the error looks like you're actually wanting @example.org but it cannot find that because you explicitly generated for sub.example.org via the setup config dkim command.

simonwiles commented 1 year ago

Thank you so much for your reply. I think my problem stems from the fact that my mailserver FQDN is sub.example.org and I am trying to configure email addresses @sub.example.org -- is it possible this is an (edge-)case that DMS isn't handling well?

I've just finished ripping everything down and checking that I can get it to work properly starting from scratch with the mailserver at sub.example.org handling mail for addresses @some-other-domain.org, and everything worked great right away[1].

In the interests of simplicity (as a non-expert here!) I'll reconfigure my infra. to remove the need for addresses @ the mailservers own FQDN; seems I'll have a happier time that way :)

[1] Which, of course, is a remarkable testament to the DMS project and all the awesome work you are doing, and I am very grateful indeed!

Thank you!

polarathene commented 1 year ago

is it possible this is an (edge-)case that DMS isn't handling well?

Absolutely! I don't have a past issue to provide but I recall one that also had this requirement.

Unfortunately there is some legacy code that tried to be helpful with the assumption of mail.example.com for DMS FQDN, and @example.com for mail. This will likely be addressed in a future major release as a breaking change fix, but it's probably going to take a while.

Usually IIRC, this should have some log about Postfix complaining (like this FAQ entry shows), and while that is advice for bare-domain setups, it might be relevant to yours as well.

Internal system mail is configured in Postfix for the standard mail.example.com, so it sees any @mail.example.com address as being managed by Postfix separately from our virtual alias/domains that are delegated to Dovecot (@example.com). So when you add accounts with setup email add ... (or provision elsewhere, such as via LDAP), this would overlap the mail.example.com domain and cause problems. I kind of cover this same information here.

For you it's a bit different, you're using LDAP which straight-up empties mydestination:

https://github.com/docker-mailserver/docker-mailserver/blob/3d5f6aeec40ac1c719becdb031fe306f33bcfcc7/target/scripts/startup/setup.d/ldap.sh#L84

This was a "fix" contributed by someone years ago and appears like it wasn't really needed specifically for LDAP, but by someone who hit the same overlap problem that the linked FAQ docs entry describes. It'll be reverted in future for consistency.

Regardless, that isn't the only case where you'll experience this issue with supporting mail accounts using addresses where the domain-part has a subdomain, especially when they overlap with the FQDN assigned to DMS (hostname). This gotcha is probably more problematic than the bare-domain configuration, since Postfix using $mydomain may not match your preferred subdomain, and I know at least with the related main.cf config setting for that SASL login behaviour with postfix appending the domain-part implicitly, it was originally configured for $myhostname, but a contributor "fixed" it with $mydomain...

As you can see over the years users have contributed "fixes" that got through review, but was really a config/documentation issue, so fixing for one group of users and breaking for another. It is my hope to get our test suite into better shape so that it can provide more confidence to maintainers and the reviewing process to prevent that sort of thing in future, but it is not a quick effort 😅

There has been some efforts to rework this stuff in the past a bit too eagerly, and I've been blocking such efforts until our test suite can better communicate we don't repeat the same mistakes.

In the interests of simplicity (as a non-expert here!) I'll reconfigure my infra. to remove the need for addresses @ the mailservers own FQDN; seems I'll have a happier time that way :)

Yeah, that'd be the best approach for now. DMS should be working well for the expected configuration of mail.example.com for DMS hostname and mail accounts using @example.com (best supported choice) or a domain-part that isn't shared with the DMS FQDN.

It should be ok to handle @sub.another-example.com since no internal configs would accidentally get mixed up.

github-actions[bot] commented 1 year ago

This issue has become stale because it has been open for 20 days without activity. This issue will be closed in 10 days automatically unless:

a maintainer removes the meta/stale label or adds the stale-bot/ignore label
new activity occurs, such as a new comment

github-actions[bot] commented 1 year ago

This issue was closed due to inactivity.

docker-mailserver / docker-mailserver