Cyrus 3.2 fails to get CRL when empty

[guimard]

From Debian#989553:

After upgrading to bullseye (from 3.0.8 to 3.2.6+CVE-2021-32056.patch) clients using tls cannot connect due to:

Jun 06 13:41:06 alpha1 cyrus/imapsr[2393]: inittls: Loading hard-coded DH parameters
Jun 06 13:41:06 alpha1 cyrus/imapsr[2388]: verify error:num=3:unable to get certificate CRL
Jun 06 13:41:06 alpha1 cyrus/imapsr[2388]: imaps TLS negotiation failed: android3.centauri.home [10.21.2.203]

This happens with both, an empty crl.pem and one with a test certificate:

root@alpha1:~# ls -l /etc/ssl
total 20
drwxr-xr-x 1 root root     10826 2021-01-24 17:54 certs
-rw-r--r-- 1 root root         0 2021-06-07 10:34 crl.pem
-rw-r--r-- 1 root root       593 2014-08-27 10:47 crl.pem.bak
-rw-r--r-- 1 root root     11943 2020-02-22 12:55 openssl.cnf
drwx--x--- 1 root ssl-cert    68 2020-12-28 16:16 private
-rw-r--r-- 1 root root        18 2018-04-01 18:14 README-crl

The relevant imapd.conf contains the following tls related stuff:

tls_server_cert: /etc/ldap/servercrt.pem
tls_server_key: /etc/ldap/serverkey.pem
tls_client_ca_file: /etc/ldap/cacert.pem
tls_session_timeout: 1440
tls_ciphers: TLSv1.2:+TLSv1:+HIGH:!aNULL:@STRENGTH
tls_versions: tls1_2 tls1_3
tls_require_cert: true
tls_crl_file: /etc/ssl/crl.pem

Commenting out the "tls_crl_file" statement lets clients connect again, but this would disable certificate revocation.

Jun 07 15:16:29 alpha1 cyrus/imapsr[3112]: login: android3.centauri.home [10.21.2.203] internet EXTERNAL+TLS User logged in SESSIONID=<cyrus-1623071789-3112-1-1100243944884731647>
Jun 07 15:16:29 alpha1 cyrus/imapsr[3135]: inittls: Loading hard-coded DH parameters
Jun 07 15:16:29 alpha1 cyrus/imapsr[3135]: starttls: TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits new) authenticated as internet

[elliefm]

The CRL support was added in #1155 (new in 3.2) and doesn't seem to have been changed since. I don't know if it's been broken somehow over time, or if it never really worked; we don't seem to have any tests for it.

I don't know what the expected format of this file is. We document this option as:

tls_crl_file: Path to a file containing the Certificate Revocation List

which I assume means something to people who know about these things, but I currently don't.

I'm skimming over the code visually, and comparing against the OpenSSL docs, and as near as I can tell it looks sensible.

We'll probably need to try and reproduce the issue with some kind of test, so that we can then add debug logging to figure out what's going awry. Since the problem apparently reproduces even with an empty CRL file, we might not need a sample one for testing, though it might still be helpful to have a sensible sample for verifying that "empty file" and "not-empty file" are not two different problems.

[elliefm]

Labelling the later versions too since this code hasn't changed since it was added, so is probably affecting all versions that have it

[karagian]

I haven't tested any of this, just noting what I know about crls... A crl file is not supposed to be empty or even contain a certificate. It contains a list of the serial numbers of the certificates that have been revoked. (certificate revocation list) The file is signed by the CA that has issued the certificates and can be used to verify a certificate. A revoked certificate is supposed to fail verification and the only way to check if the certificate has been revoked is accessing this list. If such a file is configured and it is found empty or invalid (if the file contains a certificate), I think it is normal to fail TLS...

[guimard]

Empty CRL means signed file with no serial number, not empty file

[elliefm]

The directory listing in the Debian bug report looks like they're trying to use a zero-byte file:

-rw-r--r-- 1 root root         0 2021-06-07 10:34 crl.pem
-rw-r--r-- 1 root root       593 2014-08-27 10:47 crl.pem.bak

And the timestamp on the "other" one (crl.pem.bak) is 2014?! I guess this might be a 10-year certificate that they've copied into place.

Based on recent comments that a CRL file does not contain certificates, but is rather a signed list of revoked certificate signatures, and that a CRL file containing no revoked signatures is still a signed file and not zero bytes, it sounds like the user might be misunderstanding CRLs and setting it up wrong? And in that case it sounds like the verification error is expected.

I'm on buster and apt-file doesn't find any package that provides a "crl.pem" file. Is there such a package on bullseye?

[elliefm]

Ah, looks like the way to manage CA-provided CRLs on Debian is with the fetch-crl package.

But at this point I'm assuming this person is using a variety of self-signed certificates for testing, which means fetch-crl won't help, and they'll need to create their own CRL, which I do not yet know how to do...

The openssl crl command can be used to examine (but not create) CRL files. If they run something like openssl crl -in /etc/ssl/crl.pem.bak -text -noout I think they will be able to see whether their file is valid or not, and if it's valid, what's in it. If it's not valid, then it's expected for Cyrus to fail in the way it does. And I expect the zero-byte "/etc/ssl/crl.pem" to fail in the way it does, because it's a zero byte file rather than a CRL file with no revocations.

At this point it sounds like they're trying to use something that is not a CRL file, and that Cyrus is behaving correctly by complaining. If they can produce a valid CRL file and show that the problem still occurs, then we might have a Cyrus bug, but as far as I can tell at the moment Cyrus seems fine.

---

On the subject of regression tests, in order to have any, we would need to:

• Create a self-signed CA certificate
• Create two certificates signed by our CA
• Revoke one of them
• Replace data/certs/*.pem (cassandane) and cunit/*.pem (cyrus-imapd) with the new files
• Actually write tests that try things using both certificates and expect the revoked one to fail

which is probably worth doing at some point, but a huge nuisance (we only just regenerated all our test certs last year, we shouldn't need to do so again until 2030...)

[j-pfennig]

Hi, I am the person who reported the bug at Debian

(1) I can still reproduce the problem with 1.1 empty and 1.2 the real crl (2) My crl was generated in 2014 using openssl on Debian (3) It is really that old, I created (and successfully tested) it for a customer. Since then (4) No phone was lost, no notebook stolen and cyrus worked all these years without any problems. (5) So nothing was changed until 2021 when Debian bullseye came into use.

Here is the output of openssl crl -in /etc/ssl/crl.pem.bak -text -noout

Certificate Revocation List (CRL): Version 1 (0x0) Signature Algorithm: md5WithRSAEncryption Issuer: C = DE, ST = Hessen, L = XXXXXXX, O = XXXXXXXXXXXXXXX, OU = XXXXXXXX, CN = alpha.centauri.home, name = openvpn ca, emailAddress = info@XXXXXXXXX.de Last Update: Aug 27 08:47:58 2014 GMT Next Update: Sep 26 08:47:58 2014 GMT Revoked Certificates: Serial Number: 02 Revocation Date: Aug 27 08:47:58 2014 GMT Signature Algorithm: md5WithRSAEncryption 1c:ff:b1:ba:6d:6f:ef:38:d4:a4:be:6c:66:74:f7:7b:9c:15: 15:9f:81:32:13:89:25:9d:e6:98:96:12:07:94:75:2e:e3:c3: bf:66:40:81:a3:06:fa:31:51:70:dd:61:b0:8d:e1:ed:fb:2b: b6:59:5f:b7:46:4d:c6:f7:50:b1:e2:73:f6:90:0e:06:47:c8: 08:e5:da:30:17:cd:08:99:af:40:8d:8d:d2:d7:1d:2c:fc:c5: cd:fd:62:aa:37:ad:8f:0a:b6:b6:04:6e:8c:8f:a1:15:d0:a8: c4:48:e3:62:38:24:e8:32:61:c1:d1:e7:77:a2:f2:f1:b9:43: bd:96

The ca is indeed self-signed.

Cyrus is great and I wish to thank you for your work Jürgen

[j-pfennig]

The crl.pem was 7 years old and caused the problem. I don't fully understand the thing, but may be that openssl becomes more and more picky with each new version.

• does a crl depend on on the ca that created the revoked cert? As you see the ca in my previous post was created for openvpn and openvpn comes with its own easy-rsa. Openvpn likes to have a private ca for each vpn.

• did previous versions of openssl accept a crl without known ca? Obviously the openvpn private ca was never registered in /etc/ssl/certs.

• probably i only tested the crl only with openvn. But at least it caused no problems with cyrus until Bullseye.

• For cyrus we are using certs that are created for a self-signed ca that is registered in /etc/ssl/certs.

I was able to create a crl that is accepted by openssl in the following way:

$ openssl ca -revoke Centauri/certs/centauri_revoked_cert.pem -config centauri.cnf $ openssl ca -gencrl -out crl.pem -config centauri.cnf

$ openssl verify -CRLfile crl.pem -crl_check Centauri/certs/cyrus-internet_cert.pem Centauri/certs/cyrus-internet_cert.pem: OK

$ openssl verify -CRLfile crl.pem -crl_check Centauri/certs/centauri_revoked_cert.pem C = DE, ST = Hessen, L = xxx, O = xxx, OU = REVOKE-TEST, CN = alpha.centauri.home, emailAddress = xxx error 23 at 0 depth lookup: certificate revoked error Centauri/certs/centauri_revoked_cert.pem: verification failed

Thank you very much for the helpful discussion on github Jürgen

[guimard]

Hi @j-pfennig ,

A CRL has an expiration date, so it's normal to see it rejected. Maybe Openssl was too tolerant before. I think we could close this issue since it's probably not a bug.

Regards, Yadd

[elliefm]

Hi, I am the person who reported the bug at Debian

Hi, and welcome!

The crl.pem was 7 years old and caused the problem. I don't fully understand the thing, but may be that openssl becomes more and more picky with each new version.

Aha, that explains it then. Yes, it has been my experience that each new version of OpenSSL is stricter about what it will accept. System upgrades usually find a way to shake out something that I've configured sloppily in the past that's no longer acceptable. 😅

I was able to create a crl that is accepted by openssl in the following way:

Oh, that's very helpful, thanks so much for passing it on.