Unicon / shibboleth-idp-dockerized

A Shibboleth Identity Provider (IdP) base-image
Apache License 2.0
108 stars 54 forks source link

Java exception on first run with default options #10

Closed MrFishFinger closed 8 years ago

MrFishFinger commented 8 years ago

When running tag 3.2.1 with the default options, I get the following error message: (full log and error message: https://gist.github.com/MrFishFinger/3ac12a0c420cb1296395)

Any ideas what might cause this?

2015-12-22 16:45:29.595:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@1324409e{/idp,[file:///tmp/jetty-0.0.0.0-8443-webapp-_idp-any-7429767275020554732.dir/webinf/, file:///opt/shibboleth-identity-provider-3.2.1/webapp/],AVAILABLE}{/opt/shibboleth-idp/webapp/}
2015-12-22 16:45:29.687:INFO:oejs.ServerConnector:main: Started ServerConnector@74742430{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
Exception in thread "main" MultiException[java.io.IOException: keystore password was incorrect, java.lang.IllegalStateException: no valid keystore]
  at org.eclipse.jetty.server.Server.doStart(Server.java:347)
  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
  at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1510)
  at java.security.AccessController.doPrivileged(Native Method)
  at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1435)
  Suppressed: java.lang.IllegalStateException: no valid keystore
    at org.eclipse.jetty.util.security.CertificateUtils.getKeyStore(CertificateUtils.java:48)
    at org.eclipse.jetty.util.ssl.SslContextFactory.loadKeyStore(SslContextFactory.java:1020)
    at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:336)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
    at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:64)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
    at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
    at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:260)
    at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
    at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:244)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.server.Server.doStart(Server.java:384)
    ... 4 more
Caused by: java.io.IOException: keystore password was incorrect
  at sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2015)
  at java.security.KeyStore.load(KeyStore.java:1445)
  at org.eclipse.jetty.util.security.CertificateUtils.getKeyStore(CertificateUtils.java:52)
  at org.eclipse.jetty.util.ssl.SslContextFactory.loadKeyStore(SslContextFactory.java:1020)
  at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:336)
  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
  at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
  at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
  at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:64)
  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
  at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
  at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
  at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:260)
  at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
  at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:244)
  at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
  at org.eclipse.jetty.server.Server.doStart(Server.java:384)
  ... 4 more
jtgasper3 commented 8 years ago

Yeah, go through the whole readme... You can't run it with only the default options unless your keys use the password of "changeme" or you include the passwords in idp-secrets.properties. You'll also need to generate a .p12 for the browser based TLS communications, if you haven't already.

Otherwise you'll need to update the various jetty-ssl .xml files to use a jks file or remove crypto altogether.

MrFishFinger commented 8 years ago

ok my bad, i did not read that far into the document! :) - I will have another look at it tomorrow

jtgasper3 commented 8 years ago

Can't blame you, it's definitely a TLDR document.

ghost commented 8 years ago

Hi John,

Sorry to be a pain with this. I had this running previously with the credentials and passwords externalized like in the readme. After getting the latest image built and able to run I now get this error.

Exception in thread "main" MultiException[java.io.IOException: keystore password was incorrect, java.io.IOException: keystore password was incorrect] at org.eclipse.jetty.server.Server.doStart(Server.java:347) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1510) at java.security.AccessController.doPrivileged(Native Method) at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1435) Suppressed: java.io.IOException: keystore password was incorrect at sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2015) at java.security.KeyStore.load(KeyStore.java:1445) at org.eclipse.jetty.util.security.CertificateUtils.getKeyStore(CertificateUtils.java:52) at org.eclipse.jetty.util.ssl.SslContextFactory.loadKeyStore(SslContextFactory.java:1020) at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:336) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:64) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:260) at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81) at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:244) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.server.Server.doStart(Server.java:384) ... 4 more Caused by: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded ... 21 more Caused by: java.io.IOException: keystore password was incorrect at sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2015) at java.security.KeyStore.load(KeyStore.java:1445) at org.eclipse.jetty.util.security.CertificateUtils.getKeyStore(CertificateUtils.java:52) at org.eclipse.jetty.util.ssl.SslContextFactory.loadKeyStore(SslContextFactory.java:1020) at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:336) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:64) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:260) at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81) at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:244) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.server.Server.doStart(Server.java:384) ... 4 more Caused by: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded ... 21 more

Do I need to redo the init script to build credentials everytime the underlying unicon/shibboleth-idp image changes?

Thanks Travis

On Tue, Dec 22, 2015 at 8:57 AM MrFishFinger notifications@github.com wrote:

ok my bad, i did not read that far into the document! :) - I will have another look at it tomorrow

— Reply to this email directly or view it on GitHub https://github.com/Unicon/shibboleth-idp-dockerized/issues/10#issuecomment-166673028 .

jtgasper3 commented 8 years ago

Hi @tsschmi,

You should not and generally don't want to re-run that init script as you'll essentially be changing the identity of your IdP by creating new signing, encryption, and backchannel keys.

So what was the image tag/version this was working under? And just to confirm you are using the latest (as of this morning) 3.2.1?

jtgasper3 commented 8 years ago

@MrFishFinger and @tsschmi,

I'd recommend checking out https://github.com/UniconLabs/dockerized-idp-testbed, if you haven't already. It's a full working IDP, SP, and LDAP server that runs under docker-compose. I update the idp/Dockerfile with the version of the base image I want to test and then call docker-compose build and then docker-compose up and can watch everything spin up nicely. Browse to https://idptestbed/ (after setting up an etc/hosts file entry, and you can login with staff1 and password. ctrl+c then docker-compose rm cleans everything up to try again.

I'd recommend cloning this and make sure you can get this to work. Then you can create your setup modeled after mine. You likely won't need the ldap image if you can just point at your organizations LDAP, but the rest will fit in well.

MrFishFinger commented 8 years ago

many thanks for the pointer to the testbed John, and many thanks for creating and maintaining this repo/docker image - it makes getting a test shibboleth environment setup much less painful!

i was eventually able to build a functioning testbed using docker-compose - although i ran into a docker AUFS bug when using an Ubuntu host to build the httpd-proxy image. Switching to a CentOS docker host to build the images got me around that.

Now the fun with attempting to get this to integrate with o365 for a PoC begins... urghh

jtgasper3 commented 8 years ago

You are very welcome! Please enjoy it. My Unicon IAM team and I just completed a code review and have just a few minor tweaks to make... just minor cosmetic things, but if you find any other problems or have suggestions, then please submit them. :)

As for O365, good luck. I've done it several times for clients and it really isn't that bad. If you get stuck, you might consider looking into a Shibboleth Open Source Support program subscription (https://unicon.net/opensource/shibboleth) from Unicon or just a general Shib consulting SOW.

(Thanks for listening to the sales pitch. The Shibboleth OSS program actually funds my work to produce this work.)

ghost commented 8 years ago

I think I have my bad keystore password issues figured out. I think it had to do with enabling root passwd yesterday so I could get docker-compose to install on my machine(Still not sure why that was needed). I did this shortly before your upgrade was pushed.

The behavior I saw was that with root passwd on, I could setup external idp-secrets.properties maybe once and it would work. If i stopped and ran again it would throw the bad password error. If I renamed the mounted volume from ext-conf -> my-conf, it would work once, and then stop. After disabling root passwd, this problem seems to have been resolved.

Thanks for the help. Travis

On Tue, Dec 22, 2015 at 1:52 PM John Gasper notifications@github.com wrote:

You are very welcome! Please enjoy it. My Unicon IAM team and I just completed a code review and have just a few minor tweaks to make... just minor cosmetic things, but if you find any other problems or have suggestions, then please submit them. :)

As for O365, good luck. I've done it several times for clients and it really isn't that bad. If you get stuck, you might consider looking into a Shibboleth Open Source Support program subscription ( https://unicon.net/opensource/shibboleth) from Unicon or just a general Shib consulting SOW.

(Thanks for listening to the sales pitch. The Shibboleth OSS program actually funds my work to produce this work.)

— Reply to this email directly or view it on GitHub https://github.com/Unicon/shibboleth-idp-dockerized/issues/10#issuecomment-166740318 .

jtgasper3 commented 8 years ago

Excellent!

I'll close this out.

MrFishFinger commented 8 years ago

@jtgasper3 Thanks for the info John and its good to know that there are Shibboleth paid support options out there. If we end up implementing it in our environment, I will definitely speak to my manager about potentially getting a proper support arrangement.

Unfortunately however, Shibboleth may not actually be suitable for our requirements because its support for o365 'Email-rich client logins' is quite limited - ie no Android or modern Outlook/Lync/SkypeForBusiness support. It seems like it would only work on clients that can handle a SAML login through a webpage. Sigh..

mmoayyed commented 8 years ago

Yes, fat clients are a tad difficult to get configured.

jtgasper3 commented 8 years ago

@MrFishFinger,

Those instructions are a bit dated, while the implementation process is technically accurate, it makes some broad statements that not necessarily true anymore. The ECP/PAOS profile supports Outlook and ActiveSync (iOS and Android) just fine. Older versions of Lync do have issues with Shib, but Microsoft is implementing "ADAL" which makes SAML support first class for its thick clients. I haven't implemented a Shibboleth/Office 365 integration for a client in a while, but depending upon your thick client environment, the Shib IdP could very well still be a viable solution.

Do some Googling on "ADAL" and "office 365".

panuhorsmalahti commented 8 years ago

For anyone wondering, here's an easy way to generate the idp-browser.p12, which is required to run the service for local use (of course, the browser complains about the self-signed cert):

openssl genrsa -out idp.key 2048
openssl req -new -x509 -nodes -sha1 -days 7305 -key idp.key -out idp.crt
openssl pkcs12 -export -in idp.crt -inkey idp.key -out idp-browser.p12
CsatariGergely commented 6 years ago

For me it was not obvious from the descriptions that idp-browser.p12 is mandatory to start the service and that I have to set a password and use JETTY_BROWSER_SSL_KEYSTORE_PASSWORD.

There is a typo in the Browser-based TLS Certificate and Key description: " mounted directials directory"

After some hours of wondering and trying I have a runnig server :)

jtgasper3 commented 6 years ago

Hi @CsatariGergely ,

Thanks for the note about the typo. I've updated the readme. As for the need for the idp-browser.p12, if you'd like to propose some stronger text, then you are welcome to submit a pull request. I think the following explains it, but I'm open to suggestions:

Notables

There are a few things that implementors should be aware of.

Browser-based TLS Certificate and Key

This image expects to find the TLS certificate and key for browser based communication in /opt/shibboleth-idp/credentials/idp-browser.p12.

CsatariGergely commented 6 years ago

Hi @jtgasper3 , You're welcome. I was thinking to move the TLS Certificate generation up to the "mandatory" part, like https://github.com/Unicon/shibboleth-idp-dockerized/pull/54 .