IPA scenario install / uninstall / install fails with @pki/master repo

[flo-renaud]

With the PKI packages from @pki/master repository, the scenario ipa-server-install / uninstall / install fails.

Reproducer: ipa-server-install --domain ipa.test --realm IPA.TEST --setup-dns --auto-forwarders --auto-reverse -a Secret123 -p Secret123 -U ipa-server-install --uninstall -U

At this point the file /root/ca-agent.p12 is still present and contains the admin cert from the 1st installation (behavior has not changed).

ipa-server-install --domain ipa.test --realm IPA.TEST --setup-dns --auto-forwarders --auto-reverse -a Secret123 -p Secret123 -U The above command fails with:

[...]
Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
  [1/31]: configuring certificate server instance
  [2/31]: stopping certificate server instance to update CS.cfg
  [3/31]: backing up CS.cfg
  [4/31]: Add ipa-pki-wait-running
  [5/31]: secure AJP connector
  [6/31]: reindex attributes
  [7/31]: exporting Dogtag certificate store pin
  [8/31]: disabling nonces
  [9/31]: set up CRL publishing
  [10/31]: enable PKIX certificate path discovery and validation
  [11/31]: authorizing RA to modify profiles
  [12/31]: authorizing RA to manage lightweight CAs
  [13/31]: Ensure lightweight CAs container exists
  [14/31]: Enable lightweight CA monitor
  [15/31]: Ensuring backward compatibility
  [16/31]: starting certificate server instance
  [17/31]: configure certmonger for renewals
  [18/31]: requesting RA certificate from CA
  [19/31]: publishing the CA certificate
  [20/31]: adding RA agent as a trusted user
  [21/31]: configure certificate renewals
  [22/31]: Configure HTTP to proxy connections
  [23/31]: updating IPA configuration
  [24/31]: enabling CA instance
  [25/31]: importing IPA certificate profiles
  [error] NetworkError: cannot connect to 'https://server.ipa.test:8443/ca/rest/account/login': [SSL: SSLV3_ALERT_BAD_CERTIFICATE] sslv3 alert bad certificate (_ssl.c:2559)
cannot connect to 'https://server.ipa.test:8443/ca/rest/account/login': [SSL: SSLV3_ALERT_BAD_CERTIFICATE] sslv3 alert bad certificate (_ssl.c:2559)
The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information

With PKI 11.4.3-2, the presence of /root/ca-agent.p12 was not an issue and the re-installation used to succeed, overwriting /root/ca-agent.p12. With PKI from the copr repo, we can see this new message in /var/log/pki/pki-ca-spawn.log and the re-installation fails:

2024-05-15 08:49:38 INFO: Importing admin cert from /root/ca-agent.p12
2024-05-15 08:49:38 DEBUG: Command: pki -d /root/.dogtag/pki-tomcat/ca/alias -C /root/.dogtag/pki-tomcat/ca/password.conf pkcs12-import --pkcs12 /root/ca-agent.p12 --password-file /tmp/tmp32xewit8/password.txt --debug

[amore17]

In latest run server installation is failing with error : test_pkinit_manage report Certificate issuance failed (CA_UNREACHABLE: Error 56 connecting to https://master.ipa.test:8443/ca/agent/ca//profileReview: Failure when receiving data from the peer.) DEBUG ipatests.pytest_ipa.integration.host.Host.master.cmd44:transport.py:557 The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information

[edewata]

@flo-renaud The behavior in PKI has changed recently. If the admin cert is provided (in a PKCS #12 file), pkispawn will use it instead of generating a new one. We might be able to fix it in PKI by checking whether the provided admin cert was issued by the CA being installed. If it's not, that means it might be an admin cert from an old installation, so pkispawn will overwrite it with a new one. Would that work?

@amore17 The CA_UNREACHABLE seems to be a different issue. If it keeps happening could you open a separate ticket?

[flo-renaud]

@edewata we are seeing a failure even earlier now:

Configuring certificate server (pki-tomcatd). Estimated time: 3 minutes
  [1/32]: configuring certificate server instance
  [2/32]: stopping certificate server instance to update CS.cfg
  [3/32]: backing up CS.cfg
  [4/32]: Add ipa-pki-wait-running
  [5/32]: secure AJP connector
  [6/32]: reindex attributes
  [7/32]: exporting Dogtag certificate store pin
  [8/32]: disabling nonces
  [9/32]: set up CRL publishing
  [10/32]: enable PKIX certificate path discovery and validation
  [11/32]: authorizing RA to modify profiles
  [12/32]: authorizing RA to manage lightweight CAs
  [13/32]: Ensure lightweight CAs container exists
  [14/32]: Enable lightweight CA monitor
  [15/32]: Ensuring backward compatibility
  [16/32]: starting certificate server instance
  [17/32]: configure certmonger for renewals
  [18/32]: requesting RA certificate from CA
  [error] CalledProcessError: CalledProcessError(Command ['/usr/bin/openssl', 'pkcs12', '-nokeys', '-clcerts', '-in', '/root/ca-agent.p12', '-out', '/var/lib/ipa/tmpkts4s6xd', '-passin', 'file:/tmp/tmpbfpmqn1u'] returned non-zero exit status 1: 'Can\'t open "/root/ca-agent.p12" for reading, No such file or directory\n80C27835427F0000:error:80000002:system library:BIO_new_file:No such file or directory:crypto/bio/bss_file.c:67:calling fopen(/root/ca-agent.p12, rb)\n80C27835427F0000:error:10000080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:75:\n')
CalledProcessError(Command ['/usr/bin/openssl', 'pkcs12', '-nokeys', '-clcerts', '-in', '/root/ca-agent.p12', '-out', '/var/lib/ipa/tmpkts4s6xd', '-passin', 'file:/tmp/tmpbfpmqn1u'] returned non-zero exit status 1: 'Can\'t open "/root/ca-agent.p12" for reading, No such file or directory\n80C27835427F0000:error:80000002:system library:BIO_new_file:No such file or directory:crypto/bio/bss_file.c:67:calling fopen(/root/ca-agent.p12, rb)\n80C27835427F0000:error:10000080:BIO routines:BIO_new_file:no such file:crypto/bio/bss_file.c:75:\n')
The ipa-server-install command failed. See /var/log/ipaserver-install.log for more informationExit code: 1

seen with dogtag-pki-server-11.6.0-0.1.alpha1.20240712165944UTC.dec96774.fc40.noarch