search

pki-bot / pki-issues-final

0 stars 0 forks source link

Installation of ipa with external CA fails #2711

Closed pki-bot closed 4 years ago

pki-bot commented 4 years ago

This issue was migrated from Pagure Issue #3162. Originally filed by frenaud (@flo-renaud) on 2020-03-16 13:33:31:

The 2nd step of IPA server installation fails when using the nightly build of pki from the copr repo @pki/master. The issue was detected on IPA regression tests in this PR 153 with the following logs:

self = <ipatests.test_integration.test_external_ca.TestExternalCA object at 0x7f6a88312410>

    def test_external_ca(self):
        # Step 1 of ipa-server-install.
        result = install_server_external_ca_step1(
            self.master, extra_args=['--external-ca-type=ms-cs']
        )
        assert result.returncode == 0

        # check CSR for extension
        ipa_csr = self.master.get_file_contents(paths.ROOT_IPA_CSR)
        check_mscs_extension(ipa_csr, ipa_x509.MSCSTemplateV1(u'SubCA'))

        # Sign CA, transport it to the host and get ipa a root ca paths.
        root_ca_fname, ipa_ca_fname = tasks.sign_ca_and_transport(
            self.master, paths.ROOT_IPA_CSR, ROOT_CA, IPA_CA)

        # Step 2 of ipa-server-install.
        result = install_server_external_ca_step2(
>           self.master, ipa_ca_fname, root_ca_fname)

test_integration/test_external_ca.py:145: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
test_integration/test_external_ca.py:93: in install_server_external_ca_step2
    cmd = host.run_command(args, raiseonerr=raiseonerr)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ipatests.pytest_ipa.integration.host.Host master.ipa.test (master)>
argv = ['ipa-server-install', '-U', '-r', 'IPA.TEST', '-a', 'Secret.123', ...]
set_env = True, stdin_text = None, log_stdout = True, raiseonerr = True
cwd = None, bg = False, encoding = 'utf-8', ok_returncode = 0

    def run_command(self, argv, set_env=True, stdin_text=None,
                    log_stdout=True, raiseonerr=True,
                    cwd=None, bg=False, encoding='utf-8', ok_returncode=0):
        """Wrapper around run_command to log stderr on raiseonerr=True

        :param ok_returncode: return code considered to be correct,
                              you can pass an integer or sequence of integers
        """
        result = super().run_command(
            argv, set_env=set_env, stdin_text=stdin_text,
            log_stdout=log_stdout, raiseonerr=False, cwd=cwd, bg=bg,
            encoding=encoding
        )
        # in FIPS mode SSH may print noise to stderr, remove the string
        # "FIPS mode initialized" + optional newline.
        result.stderr_bytes = FIPS_NOISE_RE.sub(b'', result.stderr_bytes)
        try:
            result_ok = result.returncode in ok_returncode
        except TypeError:
            result_ok = result.returncode == ok_returncode
        if not result_ok and raiseonerr:
            result.log.error('stderr: %s', result.stderr_text)
            raise subprocess.CalledProcessError(
                result.returncode, argv,
>               result.stdout_text, result.stderr_text
            )
E           subprocess.CalledProcessError: Command '['ipa-server-install', '-U', '-r', 'IPA.TEST', '-a', 'Secret.123', '-p', 'Secret.123', '--external-cert-file', '/ipatests/ipa_ca.crt', '--external-cert-file', '/ipatests/root_ca.crt']' returned non-zero exit status 1.

pytest_ipa/integration/host.py:193: CalledProcessError
 -----------------------------Captured stdout setup------------------------------ 
<ipatests.pytest_ipa.integration.config.Config object at 0x7f6a8824c5d0>

 -------------------------------Captured log setup------------------------------- 
INFO     ipatests.pytest_ipa.integration:__init__.py:267 Preparing host client0.ipa.test
INFO     paramiko.transport:transport.py:1819 Connected (version 2.0, client OpenSSH_8.1)
DEBUG    ipatests.pytest_ipa.integration.host.Host.client0.ParamikoTransport:transport.py:247 Authenticating with private RSA key using user root
INFO     paramiko.transport:transport.py:1819 Authentication (publickey) successful!
INFO     ipatests.pytest_ipa.integration.host.Host.client0.ParamikoTransport:transport.py:318 RUN ['true']
DEBUG    ipatests.pytest_ipa.integration.host.Host.client0.cmd1:transport.py:513 RUN ['true']
DEBUG    ipatests.pytest_ipa.integration.host.Host.client0.cmd1:transport.py:558 -bash: line 1: cd: /ipatests: No such file or directory
DEBUG    ipatests.pytest_ipa.integration.host.Host.client0.cmd1:transport.py:217 Exit code: 0
INFO     ipatests.pytest_ipa.integration:__init__.py:261 Adding client0.ipa.test:/ipatests/env.sh to list of logs to collect
DEBUG    ipatests.pytest_ipa.integration.host.Host.client0.ParamikoTransport:transport.py:301 STAT /ipatests
INFO     paramiko.transport.sftp:sftp.py:158 [chan 1] Opened sftp connection (server version 3)
DEBUG    ipatests.pytest_ipa.integration.host.Host.client0.ParamikoTransport:transport.py:301 STAT /
INFO     ipatests.pytest_ipa.integration.host.Host.client0.ParamikoTransport:transport.py:312 MKDIR /ipatests
INFO     ipatests.pytest_ipa.integration.host.Host.client0.ParamikoTransport:transport.py:293 WRITE /ipatests/env.sh
INFO     ipatests.pytest_ipa.integration:__init__.py:267 Preparing host replica0.ipa.test
INFO     paramiko.transport:transport.py:1819 Connected (version 2.0, client OpenSSH_8.1)
DEBUG    ipatests.pytest_ipa.integration.host.Host.replica0.ParamikoTransport:transport.py:247 Authenticating with private RSA key using user root
INFO     paramiko.transport:transport.py:1819 Authentication (publickey) successful!
INFO     ipatests.pytest_ipa.integration.host.Host.replica0.ParamikoTransport:transport.py:318 RUN ['true']
DEBUG    ipatests.pytest_ipa.integration.host.Host.replica0.cmd1:transport.py:513 RUN ['true']
DEBUG    ipatests.pytest_ipa.integration.host.Host.replica0.cmd1:transport.py:558 -bash: line 1: cd: /ipatests: No such file or directory
DEBUG    ipatests.pytest_ipa.integration.host.Host.replica0.cmd1:transport.py:217 Exit code: 0
INFO     ipatests.pytest_ipa.integration:__init__.py:261 Adding replica0.ipa.test:/ipatests/env.sh to list of logs to collect
DEBUG    ipatests.pytest_ipa.integration.host.Host.replica0.ParamikoTransport:transport.py:301 STAT /ipatests
INFO     paramiko.transport.sftp:sftp.py:158 [chan 1] Opened sftp connection (server version 3)
DEBUG    ipatests.pytest_ipa.integration.host.Host.replica0.ParamikoTransport:transport.py:301 STAT /
INFO     ipatests.pytest_ipa.integration.host.Host.replica0.ParamikoTransport:transport.py:312 MKDIR /ipatests
INFO     ipatests.pytest_ipa.integration.host.Host.replica0.ParamikoTransport:transport.py:293 WRITE /ipatests/env.sh
INFO     ipatests.pytest_ipa.integration:__init__.py:267 Preparing host master.ipa.test
INFO     paramiko.transport:transport.py:1819 Connected (version 2.0, client OpenSSH_8.1)
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.ParamikoTransport:transport.py:247 Authenticating with private RSA key using user root
INFO     paramiko.transport:transport.py:1819 Authentication (publickey) successful!
INFO     ipatests.pytest_ipa.integration.host.Host.master.ParamikoTransport:transport.py:318 RUN ['true']
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd1:transport.py:513 RUN ['true']
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd1:transport.py:558 -bash: line 1: cd: /ipatests: No such file or directory
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.cmd1:transport.py:217 Exit code: 0
INFO     ipatests.pytest_ipa.integration:__init__.py:261 Adding master.ipa.test:/ipatests/env.sh to list of logs to collect
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.ParamikoTransport:transport.py:301 STAT /ipatests
INFO     paramiko.transport.sftp:sftp.py:158 [chan 1] Opened sftp connection (server version 3)
DEBUG    ipatests.pytest_ipa.integration.host.Host.master.ParamikoTransport:transport.py:301 STAT /
INFO     ipatests.pytest_ipa.integration.host.Host.master.ParamikoTransport:transport.py:312 MKDIR /ipatests
INFO     ipatests.pytest_ipa.integration.host.Host.master.ParamikoTransport:transport.py:293 WRITE /ipatests/env.sh
 ------------------------------Captured stderr call------------------------------ 

[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [18/29]: authorizing RA to manage lightweight CAs
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [19/29]: Ensure lightweight CAs container exists
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [20/29]: configure certificate renewals
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [21/29]: Configure HTTP to proxy connections
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [22/29]: restarting certificate server
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [23/29]: updating IPA configuration
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [24/29]: enabling CA instance
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [25/29]: migrating certificate profiles to LDAP
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [26/29]: importing IPA certificate profiles
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [27/29]: adding default CA ACL
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [28/29]: adding 'ipa' CA entry
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [29/29]: configuring certmonger renewal for lightweight CAs
[ipatests.pytest_ipa.integration.host.Host.master.cmd16] Done configuring certificate server (pki-tomcatd).
[ipatests.pytest_ipa.integration.host.Host.master.cmd16] Configuring directory server (dirsrv)
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [1/3]: configuring TLS for DS instance
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [2/3]: adding CA certificate entry
[ipatests.pytest_ipa.integration.host.Host.master.cmd16]   [3/3]: restarting directory server
[ipatests.pytest_ipa.integration.host.Host.master.cmd16] Done configuring directory server (dirsrv).
[ipatests.pytest_ipa.integration.host.Host.master.cmd16] CalledProcessError(Command ['/bin/systemctl', 'start', 'pki-tomcatd@pki-tomcat.service'] returned non-zero exit status 1: 'Job for pki-tomcatd@pki-tomcat.service failed because a timeout was exceeded.\nSee "systemctl status pki-tomcatd@pki-tomcat.service" and "journalctl -xe" for details.\n')
[ipatests.pytest_ipa.integration.host.Host.master.cmd16] The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information
[ipatests.pytest_ipa.integration.host.Host.master.cmd16] Exit code: 1
ipa: ERROR: stderr: Checking DNS domain ipa.test., please wait ...
Checking DNS domain 122.168.192.in-addr.arpa., please wait ...
Checking DNS domain 122.168.192.in-addr.arpa., please wait ...
CalledProcessError(Command ['/bin/systemctl', 'start', 'pki-tomcatd@pki-tomcat.service'] returned non-zero exit status 1: 'Job for pki-tomcatd@pki-tomcat.service failed because a timeout was exceeded.\nSee "systemctl status pki-tomcatd@pki-tomcat.service" and "journalctl -xe" for details.\n')
The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information

Initial investigations show that the CA subsystem was unable to start, due to a failing connection to the LDAP server. The machine logs are available here and the full list of installed packages there: 389-ds-base-1.4.2.8-3.fc31.x86_64 freeipa-server-4.9.0.dev-0.fc31.x86_64 jss-4.6.3-1.20200305193355.56560f00.fc31.x86_64 pki-server-10.9.0-0.1.20200306213601.08f86385.fc31.noarch tomcatjss-7.4.1-2.20190822064831.35a49037.fc32.noarch

Note that the same tests were successful the previous week with: 389-ds-base-1.4.2.8-3.fc31.x86_64 jss-4.6.3-1.20200226223702.f580a535.fc31.x86_64 pki-server-10.9.0-0.1.20200229081402.e426a0c0.fc31.noarch tomcatjss-7.4.1-2.20190822064831.35a49037.fc32.noarch

pki-bot commented 4 years ago

Comment from cipherboy (@cipherboy) at 2020-03-16 14:44:54

Nothing relevant changed in JSS:

pki-bot commented 4 years ago

Comment from cipherboy (@cipherboy) at 2020-03-16 14:44:56

Metadata Update from @cipherboy:

pki-bot commented 4 years ago

Comment from cipherboy (@cipherboy) at 2020-03-16 14:46:16

389-ds-base and tomcatjss didn't change versions, so that leaves pki-server changes.

pki-bot commented 4 years ago

Comment from msauton (@msauton) at 2020-03-16 14:50:54

hopefully matching log details:

http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/d0430f7a-6179-11ea-9e28-fa163e8722a0/test_integration-test_external_ca.py-TestExternalCA-test_external_ca/master.ipa.test/var/log/pki/pki-tomcat/ca/debug.2020-03-08.log.gz ... 2020-03-08 20:33:51 [main] SEVERE: LdapBoundConnFactory: Unable to connect to LDAP server: Authentication failed netscape.ldap.LDAPException: Authentication failed (48) at netscape.ldap.LDAPSaslBind.checkForSASLBindCompletion(Unknown Source) at netscape.ldap.LDAPSaslBind.bind(Unknown Source) ...

and http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/d0430f7a-6179-11ea-9e28-fa163e8722a0/test_integration-test_external_ca.py-TestExternalCA-test_external_ca/master.ipa.test/var/log/dirsrv/slapd-IPA-TEST/access.gz ... [08/Mar/2020:20:33:50.736401221 +0000] conn=3 fd=65 slot=65 SSL connection from 192.168.122.112 to 192.168.122.112 [08/Mar/2020:20:33:51.415443263 +0000] conn=3 TLS1.2 128-bit AES-GCM [08/Mar/2020:20:33:51.436770269 +0000] conn=3 op=1 BIND dn="" method=sasl version=3 mech=EXTERNAL [08/Mar/2020:20:33:51.438481606 +0000] conn=3 op=1 RESULT err=48 tag=97 nentries=0 etime=0.024033785

the LDAP error code 68 is INAPPROPRIATE_AUTHENTICATION the SSL client certificate was not provided.

I could not locate the CA.cfg file, but usually, a CA is configured like this example: internaldb.ldapauth.authtype=SslClientAuth internaldb.ldapauth.bindDN=cn=Directory Manager internaldb.ldapauth.bindPWPrompt=internaldb internaldb.ldapauth.clientCertNickname=subsystemCert cert-pki-ca internaldb.ldapconn.host=ipaserver1.idm.example.test internaldb.ldapconn.port=636 internaldb.ldapconn.secureConn=true

either the CA's config is different, or the CA's LDAP connection behavior is now different.

pki-bot commented 4 years ago

Comment from cipherboy (@cipherboy) at 2020-03-16 15:24:45

CS.cfg in this case hcryptomilk't changed since 2008:

d0f2e4efbd pki/base/ca/shared/conf/CS.cfg    (PKI Team          2008-03-18 22:36:57 +0000  812) internaldb.basedn=
d0f2e4efbd pki/base/ca/shared/conf/CS.cfg    (PKI Team          2008-03-18 22:36:57 +0000  813) internaldb.maxConns=15
d0f2e4efbd pki/base/ca/shared/conf/CS.cfg    (PKI Team          2008-03-18 22:36:57 +0000  814) internaldb.minConns=3
d0f2e4efbd pki/base/ca/shared/conf/CS.cfg    (PKI Team          2008-03-18 22:36:57 +0000  815) internaldb.ldapauth.authtype=BasicAuth
d0f2e4efbd pki/base/ca/shared/conf/CS.cfg    (PKI Team          2008-03-18 22:36:57 +0000  816) internaldb.ldapauth.bindDN=cn=Directory Manager
eebdf04859 base/ca/shared/conf/CS.cfg.in     (Ade Lee           2015-04-23 13:47:24 -0400  817) internaldb.ldapauth.bindPWPrompt=internaldb
d0f2e4efbd pki/base/ca/shared/conf/CS.cfg    (PKI Team          2008-03-18 22:36:57 +0000  818) internaldb.ldapauth.clientCertNickname=
d0f2e4efbd pki/base/ca/shared/conf/CS.cfg    (PKI Team          2008-03-18 22:36:57 +0000  819) internaldb.ldapconn.host=
d0f2e4efbd pki/base/ca/shared/conf/CS.cfg    (PKI Team          2008-03-18 22:36:57 +0000  820) internaldb.ldapconn.port=
a44ccf8722 base/ca/shared/conf/CS.cfg.in     (Matthew Harmsen   2015-03-13 16:53:52 -0600  821) internaldb.ldapconn.secureConn=[PKI_DS_SECURE_CONNECTION]

The actual CA installation file is:

[CA]
pki_admin_cert_file = /root/.dogtag/pki-tomcat/ca_admin.cert
pki_admin_cert_request_type = pkcs10
pki_admin_dualkey = False
pki_admin_email = root@localhost
pki_admin_name = admin
pki_admin_nickname = ipa-ca-agent
pki_admin_password = XXXXXXXX
pki_admin_subject_dn = cn=ipa-ca-agent,O=IPA.TEST
pki_admin_uid = admin
pki_audit_group = pkiaudit
pki_audit_signing_key_algorithm = SHA256withRSA
pki_audit_signing_key_size = 2048
pki_audit_signing_key_type = rsa
pki_audit_signing_nickname = auditSigningCert cert-pki-ca
pki_audit_signing_signing_algorithm = SHA256withRSA
pki_audit_signing_subject_dn = cn=CA Audit,O=IPA.TEST
pki_audit_signing_token = internal
pki_backup_keys = True
pki_backup_password = XXXXXXXX
pki_ca_hostname = master.ipa.test
pki_ca_port = 443
pki_ca_signing_cert_path = /tmp/tmpgjfababg
pki_ca_signing_csr_path = /root/ipa.csr
pki_ca_signing_key_algorithm = SHA256withRSA
pki_ca_signing_key_size = 3072
pki_ca_signing_key_type = rsa
pki_ca_signing_nickname = caSigningCert cert-pki-ca
pki_ca_signing_record_create = True
pki_ca_signing_serial_number = 1
pki_ca_signing_signing_algorithm = SHA256withRSA
pki_ca_signing_subject_dn = CN=Certificate Authority,O=IPA.TEST
pki_ca_signing_token = internal
pki_ca_starting_crl_number = 0
pki_cert_chain_nickname = caSigningCert External CA
pki_cert_chain_path = /tmp/tmpburj6ggf
pki_client_admin_cert_p12 = /root/ca-agent.p12
pki_client_database_password = 
pki_client_database_purge = True
pki_client_dir = /root/.dogtag/pki-tomcat
pki_client_pkcs12_password = XXXXXXXX
pki_configuration_path = /etc/pki
pki_default_ocsp_uri = http://ipa-ca.ipa.test/ca/ocsp
pki_dns_domainname = ipa.test
pki_ds_base_dn = o=ipaca
pki_ds_bind_dn = cn=Directory Manager
pki_ds_database = ipaca
pki_ds_hostname = master.ipa.test
pki_ds_ldap_port = 389
pki_ds_ldaps_port = 636
pki_ds_password = XXXXXXXX
pki_ds_remove_data = True
pki_ds_secure_connection = False
pki_ds_secure_connection_ca_nickname = Directory Server CA certificate
pki_ds_secure_connection_ca_pem_file = /etc/ipa/ca.crt
pki_enable_proxy = True
pki_existing = False
pki_external = True
pki_external_pkcs12_password = 
pki_external_pkcs12_path = 
pki_external_step_two = True
pki_group = pkiuser
pki_hostname = master.ipa.test
pki_hsm_enable = False
pki_hsm_libfile = 
pki_hsm_modulename = 
pki_import_admin_cert = False
pki_instance_configuration_path = /etc/pki/pki-tomcat
pki_instance_name = pki-tomcat
pki_issuing_ca = https://master.ipa.test:443
pki_issuing_ca_hostname = master.ipa.test
pki_issuing_ca_https_port = 443
pki_issuing_ca_uri = https://master.ipa.test:443
pki_master_crl_enable = True
pki_ocsp_signing_key_algorithm = SHA256withRSA
pki_ocsp_signing_key_size = 2048
pki_ocsp_signing_key_type = rsa
pki_ocsp_signing_nickname = ocspSigningCert cert-pki-ca
pki_ocsp_signing_signing_algorithm = SHA256withRSA
pki_ocsp_signing_subject_dn = cn=OCSP Subsystem,O=IPA.TEST
pki_ocsp_signing_token = internal
pki_pkcs12_password = 
pki_pkcs12_path = 
pki_profiles_in_ldap = True
pki_random_serial_numbers_enable = False
pki_replica_number_range_end = 100
pki_replica_number_range_start = 1
pki_replication_password = 
pki_request_number_range_end = 10000000
pki_request_number_range_start = 1
pki_restart_configured_instance = False
pki_san_for_server_cert = 
pki_san_inject = False
pki_security_domain_hostname = master.ipa.test
pki_security_domain_https_port = 443
pki_security_domain_name = IPA
pki_security_domain_password = XXXXXXXX
pki_security_domain_user = admin
pki_self_signed_token = internal
pki_serial_number_range_end = 10000000
pki_serial_number_range_start = 1
pki_share_db = False
pki_skip_configuration = False
pki_skip_ds_verify = False
pki_skip_installation = False
pki_skip_sd_verify = False
pki_ssl_server_token = internal
pki_sslserver_key_algorithm = SHA256withRSA
pki_sslserver_key_size = 2048
pki_sslserver_key_type = rsa
pki_sslserver_nickname = Server-Cert cert-pki-ca
pki_sslserver_subject_dn = cn=master.ipa.test,O=IPA.TEST
pki_sslserver_token = internal
pki_status_request_timeout = 15
pki_subordinate = False
pki_subordinate_create_new_security_domain = False
pki_subsystem = CA
pki_subsystem_key_algorithm = SHA256withRSA
pki_subsystem_key_size = 2048
pki_subsystem_key_type = rsa
pki_subsystem_nickname = subsystemCert cert-pki-ca
pki_subsystem_subject_dn = cn=CA Subsystem,O=IPA.TEST
pki_subsystem_token = internal
pki_subsystem_type = ca
pki_theme_enable = True
pki_theme_server_dir = /usr/share/pki/common-ui
pki_token_name = internal
pki_user = pkiuser

Which doesn't specify LDAP auth -- could just be part of a multi-step installation program though. Would be nice if they archived the final CS.cfg and stuff before nuking the VMs.... @flo-renaud -- who would we talk to about getting the /etc/pki/pki-tomcat folder archived as part of this too? Would that be possible?

It looks like the relevant parts of the installer haven't been changed recently:

https://github.com/freeipa/freeipa/blame/master/ipaserver/install/dogtaginstance.py#L222-L269 https://github.com/freeipa/freeipa/blame/master/ipaserver/install/ca.py#L359-L423

So I'm at a loss to how this broke. Maybe a wider perspective is necessary here?

pki-bot commented 4 years ago

Comment from ftweedal (@frasertweedale) at 2020-03-17 21:02:54

I have reproduced the issue (or something very similar: server-install failure with external CA) and analysis is ongoing.

pki-bot commented 4 years ago

Comment from ftweedal (@frasertweedale) at 2020-03-18 05:33:43

OK, after narrowing things down I think it is related to the 2048-bit key size of external CA created by the test suite.

It is definitely 389-ds that is rejecting the subsytsem certificate (used by Dogtag to authenticate to LDAP). I don't know what changed that caused it to start rejecting the subsystem cert. I didn't see anything in 389-ds changelog. Maybe something changed in NSS or system crypto policy?

I'll create a PR soon to verify if a larger external CA key size dispels the issue. After that we will need to go deeper to find out what changed, and what the implications are for existing installations (if they happen to be installed with 2048-bit external CA).

pki-bot commented 4 years ago

Comment from cheimes (@tiran) at 2020-03-18 07:41:39

Fraser created https://github.com/freeipa/freeipa/pull/4382 to test installation with with a 3072 bit external CA. The larger key size did not resolve the issue.

I spun up a F31 test machine on 1minutetip, updated all packages and installed FreeIPA successfully with an external ca:

ipa-server-install -p Secret123 -a Secret123 -r IPA.EXAMPLE -n ipa.example --external-ca -U
python3 -m ipatests.create_external_ca
ipa-server-install --external-cert-file=/tmp/rootca.pem --external-cert-file=/tmp/ipaca.pem -U

My test installation was successful. Upon further investigation I saw that there is a tomcatjss version mismatch. My test machine has tomcatjss-7.4.1-2.fc31.noarch while the PR-CI machine has tomcatjss-7.4.1-2.20190822064831.35a49037.fc32.noarch. Could this mismatch cause the problem?

pki-bot commented 4 years ago

Comment from ftweedal (@frasertweedale) at 2020-03-18 09:31:24

The tomcatjss build mismatch is a possible issue. Maybe I have hit a different bug. It's all a bit confusing but I'll continue the investigation tomorrow. I definitely hit an issue with the same, or similar, symptoms, and using a larger external CA key size resolved it (I used 4096 though; I'll test again with 3072 tomorrow morning).

pki-bot commented 4 years ago

Comment from cipherboy (@cipherboy) at 2020-03-20 18:07:26

Could this mismatch cause the problem?

There's no material changes relevant to External CA or Client Authentication between TomcatJSS as shipped in Rawhide and the current master.

commit 35a490370949bca60dcec99e4f243e3f9767f950 (HEAD -> master, upstream/master, origin/master, origin/HEAD)
Author: Endi S. Dewata <edewata@redhat.com>
Date:   Tue Jul 16 19:54:07 2019 -0500

    Removed conflict with tomcat-native

    The spec file has been modified to remove the conflict with
    tomcat-native since it can be avoided by specifying the
    protocol class and sslImplementationName in the server.xml.

commit f229c67317b40dc6c8c808291b62185429edac36
Author: Dinesh Prasanth M K <SilleBille@redhat.com>
Date:   Thu Aug 8 17:22:50 2019 -0400

    Spec update

    Bumping min requirement for jss to 4.6.0

    Signed-off-by: Dinesh Prasanth M K <SilleBille@redhat.com>

commit fb11bcd42aed364f77cd1b07a0d3139496817dea
Author: Alexander Scheel <ascheel@redhat.com>
Date:   Fri May 31 15:20:46 2019 -0400

    Use JSSKeyManager and JSSTrustManager from JSS

    With jss-pr159 merged, we've added a KeyManager and TrustManager to the
    JSS default provider that we should use instead of the instances
    in-tree.

    Signed-off-by: Alexander Scheel <ascheel@redhat.com>

Top commit removes a spurious conflict with tomcat-native. There's no real, material conflict there.

Spec update was required to make sure we pull a new-enough JSS version to get KeyManager and TrustManager in the previous commit from JSS (older JSSes don't have those two in the JSSProvider). They're only used for the (future) SSLEngine and for validating/creating server side SSLEngine (with SunJSSE in Tomcat).

Otherwise, TomcatJSS isn't a factor.

pki-bot commented 4 years ago

Comment from edewata (@edewata) at 2020-04-08 19:51:51

There's a small typo causing DN encoding issue, which is in turn causing certificate validation issue, which is possibly causing the problem reported here. The typo has been fixed here: https://github.com/dogtagpki/pki/commit/5a3f3f734d58b755c3b9943cdd8d2bfac4852911

Please try again with the latest build in @pki/master. Thanks!

pki-bot commented 4 years ago

Comment from frenaud (@flo-renaud) at 2020-04-09 04:38:02

Hi, I re-tested with the following packages: freeipa-server built from master branch tomcatjss-7.4.1-2.fc31.noarch 389-ds-base-1.4.2.11-1.fc31.x86_64 jss-4.6.2-4.fc31.x86_64 Note: I didn't use the jss and tomcatjss versions from @pki/master because they trigger the issue #3170.

Issue reproduced with pki-server-10.9.0-0.1.20200408051941.7c970fcb.fc31.noarch, but not with pki-server-10.9.0-0.1.20200409033103.bcb160d4.fc31.noarch. This shows the fix is working.

Let's wait for freeIPA non-reg tests that are run during the week-end to confirm.

pki-bot commented 4 years ago

Comment from frenaud (@flo-renaud) at 2020-04-20 13:26:20

We are now hitting a different issue (reported in 3172):

[12/30]: configure certmonger for renewals
[13/30]: requesting RA certificate from CA
[error] RuntimeError: Certificate issuance failed (CA_REJECTED: Server at "http://master.ipa.test:8080/ca/ee/ca//profileSubmit" replied: Request 6 Rejected - Signing Algorithm Not Matched SHA256withRSA )

which prevents from checking if the fix is OK.

pki-bot commented 4 years ago

Comment from frenaud (@flo-renaud) at 2020-05-28 11:25:13

The latest run shows that the issue was fixed: PR 207 - successful run Versions: pki-base-10.9.0-0.1.20200523021925.617a3c1d.fc32.noarch tomcatjss-7.5.0-1.20200518183820.23655272.fc32.noarch jss-4.7.0-1.20200522211756.4791c10f.fc32.x86_64

Hence closing this issue

pki-bot commented 4 years ago

Comment from frenaud (@flo-renaud) at 2020-05-28 11:25:13

Metadata Update from @flo-renaud: