search

OxalisCommunity / Oxalis-AS4

PEPPOL AS4 pMode plugin for Oxalis
32 stars 25 forks source link

Oxalis-AS4 v.4.1.7 cannot read keystore #105

Closed gunnarvo closed 4 years ago

gunnarvo commented 4 years ago

After upgrading Oxalis-as4 from v.4.1.6 to v.4.1.7 on our Oxalis servers (inbound and outbound) won't start.

We are currently running: version.oxalis.as4: 4.1.6 version.java: 1.8.0_232 version.oxalis: 4.1.1 (inbound) version.oxalis: 4.1.2 (outbound)

Here is a part of the errorlog from our inbound server: 04-Mar-2020 08:15:46.643 INFO [main] org.apache.catalina.core.StandardServer.await A valid shutdown command was received via the shutdown port. Stopping the Server instance. 04-Mar-2020 08:15:46.656 SEVERE [Catalina-utility-2] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [oxalis] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@4aa775df]) and a value of type [io.opentracing.contrib.spanmanager.DefaultSpanManager.LinkedManagedSpan] (value [LinkedManagedSpan{NoopSpan}]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak. ... 08:15:54.121 [main] INFO n.d.o.c.settings.SettingsBuilder - Logging => CONFIG: logback-oxalis-server-V2.xml 08:15:54.121 [main] INFO n.d.o.c.settings.SettingsBuilder - Logging => SERVICE: logback 08:15:54.121 [main] INFO n.d.o.commons.logging.LoggingHandler - Logging service: logback Configuring Logback with configuration: /opt/tomcat9/.oxalis/logback-oxalis-server-V2.xml 2020-03-04 08:15:54,726 INFO [no.difi.oxalis.commons.filesystem.FileSystemModule] Inbound folder: /var/peppol/IN 2020-03-04 08:15:54,848 ERROR [no.difi.oxalis.commons.guice.GuiceModuleLoader] Error during reading of '/opt/tomcat9/.oxalis/oxalis-keystore-prod.jks'. no.difi.oxalis.api.lang.OxalisLoadingException: Error during reading of '/opt/tomcat9/.oxalis/oxalis-keystore-prod.jks'. at no.difi.oxalis.commons.security.CertificateModule.getKeyStore(CertificateModule.java:76) at no.difi.oxalis.commons.security.CertificateModule$$FastClassByGuice$$bb8feeb2.invoke() at com.google.inject.internal.ProviderMethod$FastClassProviderMethod.doProvision(ProviderMethod.java:264) at com.google.inject.internal.ProviderMethod.doProvision(ProviderMethod.java:173)

oxalis.conf from our inbound server: oxalis.keystore { path=oxalis-keystore-prod.jks password = ** key.alias = peppol-ap-prod key.password = ** } oxalis.path.inbound = /var/peppol/IN

logfile

oxalis.logging.config = logback-oxalis-server-V2.xml

Here is a part of the errorlog from our outbound server:

2020-03-04 11:00:46,209 DEBUG [org.apache.cxf.phase.PhaseInterceptorChain] [] Invoking handleMessage on interceptor org.apache.cxf.interceptor.MessageSenderInterceptor@28c62f6a 2020-03-04 11:00:46,215 DEBUG [org.apache.cxf.transport.https.SSLUtils] [] The location of the key store has not been set via a system parameter or through configuration so the default value of C:\Users\ola/.keystore will be used. 2020-03-04 11:00:46,215 DEBUG [org.apache.cxf.transport.https.SSLUtils] [] The key store password has not been set via a system property or through configuration, reading data from the keystore will fail. 2020-03-04 11:00:46,215 DEBUG [org.apache.cxf.transport.https.SSLUtils] [] The key password has not been set via a system property or through configuration, reading data from the keystore will fail. 2020-03-04 11:00:46,215 DEBUG [org.apache.cxf.transport.https.SSLUtils] [] The keystore type has not been set in configuration so the default value of JKS will be used. 2020-03-04 11:00:46,218 DEBUG [org.apache.cxf.resource.DefaultResourceManager] [] resolving resource <C:\Users\ola/.keystore> as stream
2020-03-04 11:00:46,219 DEBUG [org.apache.cxf.transport.https.SSLUtils] [] No default keystore C:\Users\ola/.keystore 2020-03-04 11:00:46,219 DEBUG [org.apache.cxf.transport.https.SSLUtils] [] The location of the trust store has not been set via a system parameter or through configuration so the default value of null will be used. 2020-03-04 11:00:46,219 DEBUG [org.apache.cxf.transport.https.SSLUtils] [] TRUST_STORE_PASSWORD_NOT_SET 2020-03-04 11:00:46,219 DEBUG [org.apache.cxf.transport.https.SSLUtils] [] The trust store type has not been set in configuration so the default value of JKS will be used.

oxalis.conf from our outbound server: oxalis.keystore { path=oxalis-keystore-prod.jks password=** key.alias=peppol-ap-prod key.password=** }

logfile

oxalis.logging.config = logback-oxalis.xml

Mode of operation? Specify TEST for pilot/test certificate or PRODUCTION for production (defaults to TEST)

oxalis.transformer.detector=noop

Timeout values in ms (https://github.com/difi/oxalis/blob/master/doc/configuration.adoc) 10 seconds OpenTimout and 2,5 minutes read/socket

oxalis.http.timeout.connect = 10000 oxalis.http.timeout.read = 150000

When switching back to oxalis-as4 v.4.1.6 the problems disappears.

Could there be a change in source code on how to read the keystore?

FrodeBjerkholt commented 4 years ago

Which version of Tomcat do you use? I am trying to reproduce the error.

FrodeBjerkholt commented 4 years ago

The main change in 4.1.7 is that it places BouncyCastle as the head SecurityProvider of the list returned by Security.getProviders()

FrodeBjerkholt commented 4 years ago

I have not been able to reproduce the error. I have tried Tomcat 8.5.50, Tomcat 9.0.12 and 9.0.30 on Windows 10 with AS 4.1.7 and oxalis 4.1.0, 4.1.1 and 4.1.2.

gunnarvo commented 4 years ago

Ok, thanks. We are running Tomcat 9.0.30 on Linux/Centos v.8 on our inbound server. Outbound we are running Windows Server 2019

Hm... the strange thing is that the error disappears when downgrading to 4.1.6. This is also independent of Oxalis version.

FrodeBjerkholt commented 4 years ago

I think the loading of the keystore depends on the SecurityProvider?

artjomsk commented 4 years ago

I also have problems with AS4 v 4.1.7 and keystore.

As I see it's because of JKS keystore usage (@gunnarvo also uses JKS).

In logs I see that keystore is being opened like PKCS12:

Caused by: java.security.UnrecoverableKeyException: Get Key failed: pad block corrupted
    at java.base/sun.security.pkcs12.PKCS12KeyStore.engineGetKey(PKCS12KeyStore.java:454) [java.base:]
    at java.base/sun.security.util.KeyStoreDelegator.engineGetKey(KeyStoreDelegator.java:90) [java.base:]
    at java.base/java.security.KeyStore.getKey(KeyStore.java:1050) [java.base:]
    at no.difi.oxalis.commons.security.CertificateModule.getPrivateKeyEntry(CertificateModule.java:89) [no.difi.oxalis-oxalis-commons-4.1.2-mrc.jar:]
    ... 113 more
Caused by: javax.crypto.BadPaddingException: pad block corrupted
    at org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher$BufferedGenericBlockCipher.doFinal(Unknown Source) [org.bouncycastle-bcprov-jdk15on-1.57.jar:1.57.0]
    at org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher.engineDoFinal(Unknown Source) [org.bouncycastle-bcprov-jdk15on-1.57.jar:1.57.0]
    at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2208) [java.base:]
    at java.base/sun.security.pkcs12.PKCS12KeyStore.lambda$engineGetKey$0(PKCS12KeyStore.java:398) [java.base:]
    at java.base/sun.security.pkcs12.PKCS12KeyStore$RetryWithZero.run(PKCS12KeyStore.java:287) [java.base:]
    at java.base/sun.security.pkcs12.PKCS12KeyStore.engineGetKey(PKCS12KeyStore.java:392) [java.base:]

No such problem with AS4 4.1.5 (haven't tried 4.1.6)

FrodeBjerkholt commented 4 years ago

I will try to debug tomorrow, to see if I can understand what is causing this.

FrodeBjerkholt commented 4 years ago

I also use a JKS keystore, but it still works.

thebilge commented 4 years ago

I also encountered the same exception. It loads fine when I downgrade to 4.1.6. I tried with different oxalis versions but seems like it is not dependent to oxalis version.

FrodeBjerkholt commented 4 years ago

The Large file fix is basicly asserting that BouncyCastle has a high enough precedence for a certain algorithm. This issue show that this breaks keystore loading for some. I will have to investigate if it is somehow possible to make it work without these side effects. Hopefully I know more tommorow.

FrodeBjerkholt commented 4 years ago

This is the java version I am using: java version "1.8.0_241" Java(TM) SE Runtime Environment (build 1.8.0_241-b07) Java HotSpot(TM) 64-Bit Server VM (build 25.241-b07, mixed mode)

FrodeBjerkholt commented 4 years ago

I have also tried: openjdk version "1.8.0_232" OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_232-b09) OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.232-b09, mixed mode)

gunnarvo commented 4 years ago

Seems like we are using mixed versions. Inbound: (Tomcat/Linux/Centos): java version: 1.8.0_232 Outbound (Windows Server 2019): java version: 1.8.0_231

But Oxalis is compiled with: java version: 1.8.0_231

I think we must upgrade to java 1.8.0_241 and use that version on all servers.

FrodeBjerkholt commented 4 years ago

Could one of you create a test keystore that fails in this way, and email it to me? The certificates doesn't need to be PEPPOL certificates. I just need to reproduce the loading error.

FrodeBjerkholt commented 4 years ago

You should also try to update Java to the latest 1.8 version. I see that there is several fixes related to keystore handling.

thebilge commented 4 years ago

I run the application with:

openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)

It also uses tomcat 9.0.16

This is the exception:

no.difi.oxalis.api.lang.OxalisLoadingException: Error during reading of '/usr/app/oxalis/oxalis-keystore.jks'.
    at no.difi.oxalis.commons.security.CertificateModule.getKeyStore(CertificateModule.java:76) ~[oxalis-commons-4.1.2.jar!/:na]
    at no.difi.oxalis.commons.security.CertificateModule$$FastClassByGuice$$bb8feeb2.invoke(<generated>) ~[oxalis-commons-4.1.2.jar!/:na]
    at com.google.inject.internal.ProviderMethod$FastClassProviderMethod.doProvision(ProviderMethod.java:264) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.ProviderMethod.doProvision(ProviderMethod.java:173) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.InternalProviderInstanceBindingImpl$CyclicFactory.provision(InternalProviderInstanceBindingImpl.java:185) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.InternalProviderInstanceBindingImpl$CyclicFactory.get(InternalProviderInstanceBindingImpl.java:162) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.ProviderMethod.doProvision(ProviderMethod.java:173) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.InternalProviderInstanceBindingImpl$CyclicFactory.provision(InternalProviderInstanceBindingImpl.java:185) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.InternalProviderInstanceBindingImpl$CyclicFactory.get(InternalProviderInstanceBindingImpl.java:162) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:52) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:147) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:124) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.BoundProviderFactory.get(BoundProviderFactory.java:60) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:211) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:182) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.Guice.createInjector(Guice.java:87) ~[guice-4.2.2.jar!/:na]
    at com.google.inject.Guice.createInjector(Guice.java:69) ~[guice-4.2.2.jar!/:na]
    at no.difi.oxalis.commons.guice.GuiceModuleLoader.initiate(GuiceModuleLoader.java:66) ~[oxalis-commons-4.1.2.jar!/:na]
    at no.difi.oxalis.as2.inbound.GuiceBeansConfig.<init>(GuiceBeansConfig.java:29) ~[classes!/:na]
    at no.difi.oxalis.as2.inbound.GuiceBeansConfig$$EnhancerBySpringCGLIB$$65912fc1.<init>(<generated>) ~[classes!/:na]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_242]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_242]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_242]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_242]
    at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:172) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:117) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:300) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:285) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1325) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1171) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:392) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1305) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1144) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:204) ~[spring-beans-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.boot.web.servlet.ServletContextInitializerBeans.getOrderedBeansOfType(ServletContextInitializerBeans.java:235) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.web.servlet.ServletContextInitializerBeans.getOrderedBeansOfType(ServletContextInitializerBeans.java:226) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.web.servlet.ServletContextInitializerBeans.addServletContextInitializerBeans(ServletContextInitializerBeans.java:101) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.web.servlet.ServletContextInitializerBeans.<init>(ServletContextInitializerBeans.java:88) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.getServletContextInitializerBeans(ServletWebServerApplicationContext.java:261) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.selfInitialize(ServletWebServerApplicationContext.java:234) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.web.embedded.tomcat.TomcatStarter.onStartup(TomcatStarter.java:54) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5125) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1377) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1367) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_242]
    at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134) ~[na:1.8.0_242]
    at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:902) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:831) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1377) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1367) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_242]
    at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134) ~[na:1.8.0_242]
    at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:902) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.core.StandardService.startInternal(StandardService.java:423) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:928) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.apache.catalina.startup.Tomcat.start(Tomcat.java:455) ~[tomcat-embed-core-9.0.16.jar!/:9.0.16]
    at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:106) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.<init>(TomcatWebServer.java:86) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getTomcatWebServer(TomcatServletWebServerFactory.java:415) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getWebServer(TomcatServletWebServerFactory.java:174) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.createWebServer(ServletWebServerApplicationContext.java:181) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:154) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543) ~[spring-context-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:142) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:316) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) ~[spring-boot-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
    at com.*******.inbound.InboundApp.main(InboundApp.java:29) ~[classes!/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_242]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_242]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_242]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_242]
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) ~[**************.jar:na]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) ~[**************.jar:na]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) ~[**************.jar:na]
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) ~[**************.jar:na]
Caused by: java.io.IOException: keystore password was incorrect
    at sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2059) ~[na:1.8.0_242]
    at sun.security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:238) ~[na:1.8.0_242]
    at sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(JavaKeyStore.java:70) ~[na:1.8.0_242]
    at java.security.KeyStore.load(KeyStore.java:1445) ~[na:1.8.0_242]
    at no.difi.oxalis.commons.security.CertificateModule.getKeyStore(CertificateModule.java:70) ~[oxalis-commons-4.1.2.jar!/:na]
    ... 110 common frames omitted
Caused by: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: pad block corrupted
    ... 115 common frames omitted

Somehow it says "keystore password was incorrect" but it is not. When I only change oxalis-as4 version to 4.1.6, it starts up fine.

FrodeBjerkholt commented 4 years ago

The reason for these problems are related to the SecurityProvider. In 4.1.7 Bound Castle is the preferred one and this somehow leads to Keystore loading problems for some users.
It would be nice if someone could email me a test-keystore that triggers this problem.

FrodeBjerkholt commented 4 years ago

Good news - I have found an alternative way of specifying the preferred security provider for a given algorithm. This means that I can remove the code lines that cause this problem. It will be fixed in the 4.1.8 release.