Add "Lets Encrypt CA" Certpath to SSVM Keystore (for cdimage.debian.org)

mdesaive commented 6 years ago

ISSUE TYPE

Improvement Request

COMPONENT NAME

SSVM

CLOUDSTACK VERSION

4.11

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

The Download of ISO Images from https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso fails.

sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The site "cdimage.debian.org" uses Certificates from "Lets Encrypt".

Could it be the certification path for "Lets Encrypt" is not included in the Java Keystore on the SSVM? If so, could "Lets Encrypt" be added to the Keystore?

STEPS TO REPRODUCE

Log in to Web-UI
Register ISO
Enter Download URL "https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso"

EXPECTED RESULTS

The Debian ISO image should be installed correctly.

ACTUAL RESULTS

Download fails with:

    sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

mdesaive commented 6 years ago

Duplicates: https://issues.apache.org/jira/browse/CLOUDSTACK-10351 Closed issue in Jira.

rohityadavcloud commented 6 years ago

This is more of a debian issue than CloudStack issue, example: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=809259

On my testing, I found that debian ca-certificates is OK. Curling a letsencrypt https endpoint works but the java based agents fails for the same https URL with:

sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

We'll refresh a new systemvmtemplate as part of 4.11.1.0 release (that will be compatible with 4.11.0.0 as well) to include update ca-certificates packages. In addition, what we can do is to manually install/setup letsencrypt ca certs in the systemvmtemplate from https://letsencrypt.org/certificates/

rafaelweingartner commented 6 years ago

@rhtyd do we change the default cacerts in the system VMs template? I faced this problem sometime ago; it was due to the java keystore (cacerts) that did not contain the latest Let's encrypts CA certificates.

I manually added them to the CA certs. In my case, wget/curl was also working fine because Debian certificates were updated.

rohityadavcloud commented 6 years ago

@rafaelweingartner another way exists which is to update the keystore file in systemvm.iso.

@mdesaive can you try this on your ssvm and confirm:

cd /usr/local/cloud/systemvm/certs
wget https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.der
keytool -trustcacerts -keystore realhostip.keystore -storepass vmops.com -noprompt -importcert -alias letsencryptauthorityx3cross -file lets-encrypt-x3-cross-signed.der
systemctl restart cloud

Now, try adding a template from a https url which uses letsencrypt certificate.

mdesaive commented 6 years ago

@rhtyd Thanks for the commands to update the Java keystore. Worked perfectly! And also big "thank you" for the commit!

Absolutely great, how fast you are reacting to the issues we posted! Great project! Great team!

s-seitz commented 6 years ago

@rhtyd Thank's for your response on how to hot fix this. But I'ld like to start a discussion if the SSVM wouldn't be better instanciated with recent (or updated) root CA's trusted by the project and/or the respective admins. Maybe that wget could do an initial first-boot job pointing to a configured or hardwired path on the storage?

rohityadavcloud commented 6 years ago

@s-seitz We cannot assume that systemvms may have access to public network when they start, so the best solution is to fix this during systemvmtemplate generation and ensure that the agent/java process uses the system keystore file instead of the custom/old one. This creates an issue to regression test custom ssl certificate(s) upload feature in CloudStack on both ssvm and cpvm. I'm busy this week, but I can revisit this in next 2 weeks, certainly before the next release milestone 4.11.1.0.

s-seitz commented 6 years ago

@rhtyd Great and thank's in advance if you'ld find time before 4.11.1.0. I was not thinking about downloading a truststore from public ressources instead of having it somewhere at a fixed location at the storage the SSVM is connected to.

rohityadavcloud commented 6 years ago

Better fix implemented here: https://github.com/apache/cloudstack/pull/2616 The idea is we include Letsencypt cacerts during systemvmtemplate build, and during ssvm/cpvm patching the local keystore imports the global keystore.

rohityadavcloud commented 6 years ago

Closing this as this is addressed in new 4.11.1 systemvmtemple from this pr https://github.com/apache/cloudstack/pull/2616

apache / cloudstack