search

SSSD / sssd

A daemon to manage identity, authentication and authorization for centrally-managed systems.
https://sssd.io
GNU General Public License v3.0
583 stars 234 forks source link

regression in certificate configuration/handling from NSS to OpenSSL #7486

Open qralston opened 1 month ago

qralston commented 1 month ago

When sssd is configured (at compile time) to use NSS, the pam_cert_db_path option is the path to an NSS database, defaulting to /etc/pki/nssdb (the system NSS database).

Thanks to p11-kit-proxy, the system NSS database automatically includes all certificates in the consolidated system-wide CA trust database, maintained via update-ca-trust(8). Furthermore, because NSS differentiates CA certificates that are anchors (certificates trusted as root CA certificates) versus non-anchors (intermediate certificates present in order to complete certificate chains but are not themselves anchors), as long as the system administrator installs all required root CA certificates in:

…and installs all required intermediate CA certificates in:

…then when sssd uses NSS, the default pam_cert_db_path value of /etc/pki/nssdb ensures correct certificate configuration of sssd, with sssd using the consolidated system-wide CA trust database maintained via update-ca-trust(8). (And the entire point of maintaining the consolidated system-wide CA trust database is to provide a central location where CA trust is maintained, without having to maintain CA trust information for each application on the system.)

But alas, in contrast, when sssd is compiled against OpenSSL instead of NSS, all of these advantages are lost, and only disadvantages are acquired.

First, when OpenSSL is in use, the value of the pam_cert_db_path option defaults to /etc/sssd/pki/sssd_auth_ca_db.pem. sssd does not contribute this file, so immediately, this will break smart card authentication, requiring the administrator to inspect the sssd logs to determine that sssd is attempting to open a file that does not exist.

Second, OpenSSL’s certificate “store” is simply a list of one or more certificates (PEM or PKCS#12 format). Unlike the NSS certificate store, neither the PEM nor PKCS#12 format permits differentiating certificates that are anchors versus certificates that are non-anchors (intermediates). Rather, it is up to the application to provide a configuration mechanism for the system administrator / end user to specify certificates that are anchors versus intermediates.

For example, MIT Kerberos provides a mechanism in the krb5.conf file to specify both anchor and intermediate certificates:

Because both options accept the DIR: prefix, and because both options will check all files in the specified directory for PEM-encoded certificates, it is possible to configure MIT Kerberos to use the system-wide CA trust database with this configuration:

[realms]

    EXAMPLE.ORG = {
        pkinit_anchors = DIR:/etc/pki/ca-trust/source/anchors
        pkinit_pool = DIR:/etc/pki/ca-trust/source
    }

Or:

[realms]

    EXAMPLE.ORG = {
        pkinit_anchors = FILE:/etc/pki/tls/certs/ca-bundle.crt
        pkinit_pool = DIR:/etc/pki/ca-trust/source
    }

In contrast, when compiled against OpenSSL, sssd fails to provide a mechanism to specify anchor versus non-anchor certificates. There is only the single pam_cert_db_path option.

If the system administrator attempts to set pam_cert_db_path to the ca-trust extracted PEM bundle of CA anchors (/etc/pki/tls/certs/ca-bundle.crt), smart card authentication will fail, because sssd will not have the intermediate certificates that are necessary to build the certificate chain from the smart card certificate to the corresponding root CA certificate.

So this means that pam_cert_db_path must be set to a file that contains all requisite root CA certificates and all requisite intermediate certificates. This is a problem, for two reasons:

  1. Because there is no way to differentiate anchor versus non-anchor certificates in the file specified by pam_cert_db_path, sssd will treat all non-anchor certificates in pam_cert_db_path as anchor (root CA) certificates. This is wrong, and defeats the entire point of having a CA hierarchy in the first place.
  2. update-ca-trust(8) does not maintain a file of combined anchor and non-anchor CA certificates (because, per the previous point, there would be no way to differentiate anchor versus non-anchor certificates in said file). This means it is impossible for sssd to use the consolidated system-wide CA trust database.

So, in conclusion, while the move from NSS to OpenSSL may have resulted in improvements to sssd overall, due to differing capabilities of NSS versus OpenSSL in differentiating anchor versus non-anchor certificates for smart card authentication, moving to OpenSSL necessitated additional changes to preserve the configuration and security advantages of NSS:

  1. Be capable of using the consolidated system-wide CA trust database.
  2. Moreover, default to using the consolidated system-wide CA trust database.
  3. Do not erroneously treat non-anchor CA certificates as anchors.

These issues were either not identified, or not acted upon. As a result, when sssd is configured to use OpenSSL instead of NSS:

  1. sssd is not capable of using the consolidated system-wide CA trust database.
  2. sssd does not default to using the consolidated system-wide CA trust database (implied by the previous point).
  3. sssd erroneously treats non-anchor CA certificates as anchors, which is cryptographically incorrect, and defeats the security advantages of having certificate chains in the first place.

The only way to remedy these issues is to provide additional configuration mechanisms to 1) permit the system administrator to differentiate anchor versus non-anchor certificates, and 2) do so in a way that utilizes the consolidated system-wide CA trust database. E.g.:

Making these changes would eliminate the certificate configuration/handling regressions that occurred in the move from NSS to OpenSSL.

Thoughts?

sumit-bose commented 1 month ago

Hi,

thank you very much for your detail proposal.

I'm open for adding the suggested pool and anchors options. Is there a reason you prefer _file and _dir options over a single option and using FILE: and DIR: prefixes (with FILE: as default if missing)? I'm asking because imo this would make it more clear that the options do not compete and that everything listed will be included. And it would be easier to add new types in future.

Originally I was wondering if there should be a distinction between the two similar as in krb5.conf but I decided against for a start because I've seen lots of configurations where either pkinit_anchors and pkinit_pool where either pointing to the same file or if there were different files they had the same content. So I wanted to make the configuration more easy and reused the single option which was already available. But I agree, doing it the right way requires a distinction between the anchors and the intermediates.

I'm a bit reluctant with using /etc/pki/tls/certs/ca-bundle.crt and /etc/pki/ca-trust/source as defaults. I tired to avoid the system-wide CA store because I think its main purpose is to make https access for web browsers work ideally for all web sites in the world. This requires to have lots of CA stored here and although type might be curated I wonder if you really want to trust all of them with respect to logging into your system. This might be neglectable if you use the whole certificate for mapping but if you use only attributes from the certificate the signature of a trusted is the only protection against self made certificates with the same attributes as the official certificate. So I would prefer to not use any defaults here but to make the admin's life more easy a value like .e.g SYSTEM: can tell SSSD to use the system's CA store.

bye, Sumit

qralston commented 1 month ago

Checking the sssd man pages, I didn’t see any other instances of sssd options taking FILE: or DIR: prefixes, so I hesitated to introduce them. But I think as long as the man page makes it clear that only the pam_cert_anchors and pam_cert_pool options take these prefixes, then I agree that having single options would be cleaner.

In terms of not defaulting pam_cert_anchors to the system-wide CA trust store of anchors… for our particular use case, where our Linux hosts only ever use smart card authentication against AD via PKINIT, it doesn’t matter if sssd trusts all anchors in the system-wide CA trust store: the KDCs don’t. So even if an attacker had a smart card with the correct values to satisfy the mapping rules the system administrator configured for sssd, authentication would still fail, because AD would reject the attacker’s smart card’s certificate.

Also, I was attempting to preserve the NSS behavior, which defaulted to the system-wide CA trust store.

But if it is possible to configure sssd to perform non-PKINIT smart card authentication, where sssd itself is verifying the authentication (instead of punting that to the KDCs via PKINIT), then yes; if an attacker could find someone to issue a smart card with the correct values from an (ultimate) anchor CA that is one of the ones (~142 and counting) in the system-wide CA trust store, then they’re onto the system. Even if that risk is small, it should be avoided.

For that reason, I agree it makes sense to require the administrator to explicitly configure the anchor CAs (that is, make the default unset), and additionally recommend that they include only anchor CAs that they actually use for smart card authentication.

So, based on all of that, here is a rewrite:

  • pam_cert_anchors

    A comma-separated list of one or more anchor (root CA) certificate sources to trust for smart card authentication. Each source consists of a fully-qualified file path prefixed with one of (FILE:, DIR:).

    For the FILE: prefix, the path must be a plain file or symbolic link. All PEM-encoded certificates found in the file are loaded.

    For the DIR: prefix, the path must be a directory or symbolic link to a directory. For all plain files or symbolic links in the specified directory, all PEM-encoded certificates in the file are loaded.

    The DIR: prefix attempts to load certificates from all files in the directory; there is no way to exclude files. If you wish to load only certain certificate files in a directory, specify the individual files you wish to load using a list of FILE: prefixes, instead of using the DIR: prefix on the parent directory.

    Note that it is generally a security best practice to specify only anchor CA certificates that are known to be used for smart card authentication on the system. This prevents attackers from attempting to spoof/forge user authentication with smart cards issued by other parties.

    Default: unset (no anchor root CA certificates are loaded).

    See also pam_cert_pool.

  • pam_cert_pool

    A comma-separated list of one or more non-anchor (intermediate CA) certificate sources for sssd to use in order to complete certificate chains from smart card certificates to the anchor CA certificates specified via pam_cert_anchors. Each source consists of a fully-qualified file path prefixed with one of (FILE:, DIR:).

    For the FILE: prefix, the path must be a plain file or symbolic link. All PEM-encoded certificates in the file are loaded.

    For the DIR: prefix, the path must be a directory or symbolic link to a directory. For all plain files or symbolic links in the specified directory, all PEM-encoded certificates in the file are loaded.

    Note that sssd never trusts intermediate certificates for authentication directly. sssd uses intermediate certificates specified via pam_cert_pool only to complete certificate chains from smart card certificates to the anchor CA certificates specified via pam_cert_anchors. Therefore, in contrast to pam_cert_anchors, it does no harm to widely include intermediate certificates.

    Default: DIR:/etc/pki/ca-trust/source, which loads all intermediate CA certificates in the system-wide CA trust store.

    See also pam_cert_anchors.

I’m not sure if upstream sssd even still permits compiling against NSS, or if it OpenSSL-only at this point. If the latter, then it probably makes sense to remove the pam_cert_db_path entirely, emitting a warning to use pam_cert_anchors and/or pam_cert_pool instead if it is encountered in the sssd configuration.

Thoughts?

sumit-bose commented 1 month ago

Hi,

thank you for all your input.

about "sssd itself is verifying the authentication", this will happen if the system is offline and there was a successful online authentication before. During the online authentication SSSD stored into the cache which authentication methods were offered by the server side for the given user and when offline this information is used. So even with PKINIT available local authentication might happen when offline.

DIR: and FILE: are already borrowed from krb5.conf and used in the krb5_ccname_template option, see man sssd-krb5, so I think it is ok to use them here as well, but I will check with the other developers if there are concerns.

About the DIR: type, if I understand it correctly OpenSSL does not read all files from a directory but will only read file where the name has the format hash.N, i.e. if will not read the whole directory content but will read CA certificate files when needed base on the hash of the subject DN of the CA certificate, see e.g. man X509_LOOKUP_hash_dir or man openssl-rehash. This should be mentioned in the description for DIR:. As a result I think DIR:/etc/pki/ca-trust/source is not a suitable default; additionally, /etc/pki/ca-trust/source/ can not only contain intermediate CA certificate but also anchors if the PEM file has the OpenSSL "trusted" format extension with suitable options, see man openssl-x509 for the "trusted" extension and man update-ca-trust for the usage for this directory.

As default for pam_cert_pool /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt might be suitable because it simply contains all certificates, i.e. anchors and intermediates.

About NSS, recent versions of SSSD only support OpenSSL. Nevertheless, I think pam_cert_db_path cannot be remove immediately but we should add a deprecation warning in the logs and ask to replace with the new options for a couple of releases before removing it.

bye, Sumit

qralston commented 1 month ago

Ah; good point about local authentication when offline. That’s an even better reason to define the set of anchor CA certificates minimally, instead of just trusting every anchor CA in the system-wide CA trust store.

As per the MIT Kerberos documented krb5.conf DIR: behavior, emphasis mine:

In pkinit_anchors or pkinit_pool, dirname is assumed to be an OpenSSL-style hashed CA directory where each CA cert is stored in a file named hash-of-ca-cert.#. This infrastructure is encouraged, but all files in the directory will be examined and if they contain certificates (in PEM format), they will be used.

The reason why this works for krb5.conf is because load_cas_and_crls_dir() in src/plugins/preauth/pkinit/pkinit_crypto_openssl.c calls opendir(), skips any directories and any files the begin with ., and then calls load_cas_and_crls() on every entry, which calls BIO_new_file() and then PEM_X509_INFO_read_bio() on the entry.

Is this a little bit of extra work? Yes. But the reason why this behavior is highly desirable is because update-crypto-policies(8) does not create OpenSSL hashes. Because krb5.conf does not require the hashes, the system administrator can set pkinit_pool to DIR:/etc/pki/ca-trust/source in order to configure krb5.conf to use the same intermediate certificates that are available in the system-wide CA trust store, without worrying about how to maintain the hashes.

I think replicating the behavior of the krb5.conf DIR: prefix for pkinit_anchors and pkinit_pool, as closely as possible, for the behavior of the DIR: prefix for sssd’s pam_cert_anchors and pam_cert_pool options (respectively) is the best course of action here. This ensures that the system administrator can use the same values for both (respective) options, and it avoids forcing the system administrator to manually maintain OpenSSH hashes when krb5.conf does not require this.

I agree that pam_cert_db_path shouldn’t be removed immediately, but I do wonder how many system administrators are going to see the deprecation log message if it appears only in the /var/log/sssd/*.log files. Would it be possible to log a deprecation notice to the syslog authpriv facility, so that it shows up in /var/log/secure?

sumit-bose commented 1 month ago

Hi,

I agree, since MIT Kerberos's libkrb5 is doing this we should be compatible here.

I'm still reluctant about using DIR:/etc/pki/ca-trust/source as a default because I think the intention of update-ca-trust is that only items from /etc/pki/ca-trust/extracted should be used since there might be multiple sources, e.g. /usr/share/pki/ca-trust-source/.

About /var/log/secure, yes this or the journal can be used as well.

bye, Sumit

qralston commented 1 month ago

I think it’s reasonable to default pam_cert_pool to unset if that’s what you prefer. Since the system administrator will definitely have to set pam_cert_anchors, one could make the argument that sssd should defer all CA certificate configuration to the sysadmin, rather than attempt to second-guess what the sysadmin intends.

In terms of getting the pam_cert_db_path deprecation notice in front of the sysadmins, anything that goes to the systemd journald should suffice. (Sites that run e.g. rsyslog will almost certainly configure rsyslog to pull from journald.) My main concern here is that if the only place the deprecation notice appears is in a /var/log/sssd/*.log file, the chance the sysadmins will see it is probably not that good. (I suspect most sysadmins only start looking through the /var/log/sssd/*.log files when something is going wrong.)

Again, thanks for considering this. sssd is one of the most important pieces of software we run on our RHEL systems, because it enables seamless AD user/group/authentication integration. We appreciate all of the hard work that goes into sssd!

sumit-bose commented 1 month ago

Hi,

thank you for your kind words.

I can start working on this when I'm back from PTO but feel free to create a pull-request on your own.

bye, Sumit