search

aws / aws-sdk-java

The official AWS SDK for Java 1.x (In Maintenance Mode, End-of-Life on 12/31/2025). The AWS SDK for Java 2.x is available here: https://github.com/aws/aws-sdk-java-v2/
https://aws.amazon.com/sdkforjava
Apache License 2.0
4.13k stars 2.83k forks source link

Using the AmazonSimpleEmailService triggers a SSL related error on Java 9 #1511

Closed jorn86 closed 6 years ago

jorn86 commented 6 years ago

I'm using the AWS library to send email through SES. On java 8, it works fine. Other non-SES services also work fine on java 9. But this particular combination triggers an error. Here's the relevant part of the log:

14:50:55 Emn[0BA5]  DEBUG Sending Request: POST https://email.eu-west-1.amazonaws.com / Parameters: ({"Action":["SendRawEmail"],"Version":["2010-12-01"],"Source":["noreply@teamsolar.works"],"Destinations.member.1":["j+3@worcade.com"],"RawMessage.Data":[<snip>]}Headers: (User-Agent: aws-sdk-java/1.11.289 Linux/4.10.0-42-generic OpenJDK_64-Bit_Server_VM/9.0.4+11 java/9.0.4, amz-sdk-invocation-id: e8f412ae-b50c-ac22-38fd-ef6e430e48d9, )  (AmazonHttpClient$RequestExecutor:1183)
14:50:55 Emn[0BA5]  DEBUG AWS4 Canonical Request: '"POST
/

amz-sdk-invocation-id:e8f412ae-b50c-ac22-38fd-ef6e430e48d9
amz-sdk-retry:0/0/500
host:email.eu-west-1.amazonaws.com
user-agent:aws-sdk-java/1.11.289 Linux/4.10.0-42-generic OpenJDK_64-Bit_Server_VM/9.0.4+11 java/9.0.4
x-amz-date:20180315T145055Z

amz-sdk-invocation-id;amz-sdk-retry;host;user-agent;x-amz-date
01aafc60dd85a353462f33018b2e2684748f914a96d485a911a2fe1779ac9e4c" (CommonsLog:33)
14:50:55 Emn[0BA5]  DEBUG AWS4 String to Sign: '"AWS4-HMAC-SHA256
20180315T145055Z
20180315/eu-west-1/ses/aws4_request
89256f0cfb6e1e7b338296dbced3c892fbdaca056483c2e6700a30db3e129523" (CommonsLog:33)
14:50:55 Emn[0BA5]  DEBUG Generating a new signing key as the signing key not available in the cache for the date 1521072000000                              (CommonsLog:33)
14:50:55 Emn[0BA5]  DEBUG connecting to email.eu-west-1.amazonaws.com/52.94.217.28:443                                                                       (SdkTLSSocketFactory:127)
14:50:55 Emn[0BA5]  DEBUG Connecting socket to email.eu-west-1.amazonaws.com/52.94.217.28:443 with timeout 10000                                             (SSLConnectionSocketFactory:335)
14:50:55 Emn[0BA5]  DEBUG Enabled protocols: [TLSv1, TLSv1.1, TLSv1.2]                                                                                       (SSLConnectionSocketFactory:388)
14:50:55 Emn[0BA5]  DEBUG Enabled cipher suites:[TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV] (SSLConnectionSocketFactory:389)
14:50:55 Emn[0BA5]  DEBUG socket.getSupportedProtocols(): [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2], socket.getEnabledProtocols(): [TLSv1, TLSv1.1, TLSv1.2] (SdkTLSSocketFactory:74)
14:50:55 Emn[0BA5]  DEBUG TLS protocol enabled for SSL handshake: [TLSv1.2, TLSv1.1, TLSv1]                                                                  (SdkTLSSocketFactory:102)
14:50:55 Emn[0BA5]  DEBUG Starting handshake                                                                                                                 (SSLConnectionSocketFactory:393)
14:50:56 Emn[0BA5]  DEBUG                                                                                                                                    (ClientConnectionManagerFactory$Handler:82)
java.lang.reflect.InvocationTargetException: null
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:564)
    at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76)
    at com.amazonaws.http.conn.$Proxy39.connect(Unknown Source)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
    at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1236)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
    at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
    at com.amazonaws.services.simpleemail.AmazonSimpleEmailServiceClient.doInvoke(AmazonSimpleEmailServiceClient.java:5193)
    at com.amazonaws.services.simpleemail.AmazonSimpleEmailServiceClient.invoke(AmazonSimpleEmailServiceClient.java:5169)
    at com.amazonaws.services.simpleemail.AmazonSimpleEmailServiceClient.executeSendRawEmail(AmazonSimpleEmailServiceClient.java:3692)
    at com.amazonaws.services.simpleemail.AmazonSimpleEmailServiceClient.sendRawEmail(AmazonSimpleEmailServiceClient.java:3668)
    at net.worcade.mail.sender.AmazonSesSender.send(AmazonSesSender.java:97)
    at net.worcade.mail.sender.AmazonSesSender.send(AmazonSesSender.java:66)
    at net.worcade.mail.AbstractEmailService.sendEmail(AbstractEmailService.java:134)
    at net.worcade.mail.EmailNotificationService.handleConversationInvite(EmailNotificationService.java:97)
    at net.worcade.mail.EmailNotificationService.handleRequest(EmailNotificationService.java:63)
    at net.worcade.service.rabbitmq.RabbitMqService$RetryingListener.handleInternal(RabbitMqService.java:272)
    at net.worcade.service.rabbitmq.RabbitMqService$BasicListener.handleDelivery(RabbitMqService.java:217)
    at com.rabbitmq.client.impl.ConsumerDispatcher$5.run(ConsumerDispatcher.java:149)
    at com.rabbitmq.client.impl.ConsumerWorkService$WorkPoolRunnable.run(ConsumerWorkService.java:100)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at java.base/sun.security.ssl.Alerts.getSSLException(Alerts.java:198)
    at java.base/sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1969)
    at java.base/sun.security.ssl.Handshaker.fatalSE(Handshaker.java:345)
    at java.base/sun.security.ssl.Handshaker.fatalSE(Handshaker.java:339)
    at java.base/sun.security.ssl.ClientHandshaker.checkServerCerts(ClientHandshaker.java:1968)
    at java.base/sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1777)
    at java.base/sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:264)
    at java.base/sun.security.ssl.Handshaker.processLoop(Handshaker.java:1092)
    at java.base/sun.security.ssl.Handshaker.processRecord(Handshaker.java:1026)
    at java.base/sun.security.ssl.SSLSocketImpl.processInputRecord(SSLSocketImpl.java:1137)
    at java.base/sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1074)
    at java.base/sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
    at java.base/sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1402)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1429)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353)
    at com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(SdkTLSSocketFactory.java:132)
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:141)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
    ... 37 common frames omitted
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385)
    at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:290)
    at java.base/sun.security.validator.Validator.validate(Validator.java:264)
    at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:343)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:226)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:133)
    at java.base/sun.security.ssl.ClientHandshaker.checkServerCerts(ClientHandshaker.java:1947)
    ... 52 common frames omitted
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
    at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
    at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
    at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380)
    ... 58 common frames omitted
14:50:56 Emn[0BA5]  DEBUG Retrying Request: POST https://email.eu-west-1.amazonaws.com / Parameters: ({"Action":["SendRawEmail"],"Version":["2010-12-01"],"Source":["noreply@teamsolar.works"],"Destinations.member.1":["j+3@worcade.com"],"RawMessage.Data":<snip>}Headers: (User-Agent: aws-sdk-java/1.11.289 Linux/4.10.0-42-generic OpenJDK_64-Bit_Server_VM/9.0.4+11 java/9.0.4, amz-sdk-invocation-id: e8f412ae-b50c-ac22-38fd-ef6e430e48d9, )  (AmazonHttpClient$RequestExecutor:1183)
14:50:56 Emn[0BA5]  DEBUG Retriable error detected, will retry in 66ms, attempt number: 0                                                                    (AmazonHttpClient$RequestExecutor:1666)

I'm running openjdk 9.0.4 on Ubuntu 17.04.

As a workaround, would it be possible to override the http client used by the library, e.g. from apache to Jersey? I checked, but could only find config options for apache, not a full client implementation override.

dagnir commented 6 years ago

Hi, this could be due to the OpenJDK 9 installation not containing the root Amazon Trust Services certificates. Can you check your JDK 9 installation's cacerts to ensure the root CA's listed in this blog post are there? https://aws.amazon.com/blogs/security/how-to-prepare-for-aws-move-to-its-own-certificate-authority/

Also, which services and regions are working okay for you with JDK 9? We can check to see what root CA's they're using to see if this is the issue.

jorn86 commented 6 years ago

Services: The same SSL error occurs using the SDK for Cloudsearch. The SDK for S3 works without triggering the error.

For testing the certificates, I tried opening a HttpsUrlConnection to each of the URLs mentioned in the article you linked. Those all open (new URL(url).openConnection();) just fine: For https://good.sca0a.amazontrust.com/, success For https://good.sca1a.amazontrust.com/, success For https://good.sca2a.amazontrust.com/, success For https://good.sca3a.amazontrust.com/, success For https://good.sca4a.amazontrust.com/, success For https://starfieldtech.com/, success

but when actually trying to read them (new URL(url).openConnection().getInputStream();) they throw the exception: For https://good.sca0a.amazontrust.com/, sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target For https://good.sca1a.amazontrust.com/, sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target For https://good.sca2a.amazontrust.com/, sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target For https://good.sca3a.amazontrust.com/, sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target For https://good.sca4a.amazontrust.com/, sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target For https://starfieldtech.com/, sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Hope this helps, and please let me know if you need more information

dagnir commented 6 years ago

@jorn86 So just opening the connection doesn't appear to be enough to check that your JVM can successfully validate the cert; I guess it doesn't attempt the handshake until after you try to get the InputStream.

For example, using one of the expired URLs from https://www.amazontrust.com/repository/ (such as https://expired.sca1a.amazontrust.com/), I can open the connection but I also get a cert error when try to do connection.getInputStream(). However, it works fine with the the valid certificates.

Assuming you were trying to use CloudSearch in eu-west-1 as well, it looks like they're also using the new Root CA now:

curl -v https://cloudsearch.eu-west-1.amazonaws.com
* Rebuilt URL to: https://cloudsearch.eu-west-1.amazonaws.com/
*   Trying 52.95.120.87...
* Connected to cloudsearch.eu-west-1.amazonaws.com (52.95.120.87) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
* Server certificate: cloudsearch.eu-west-1.amazonaws.com
* Server certificate: Amazon
* Server certificate: Amazon Root CA 1
* Server certificate: Starfield Services Root Certificate Authority - G2

At this point, looks like your installation is missing the root CA's. Updating your trust store should fix this issue.

dagnir commented 6 years ago

Going to go ahead and close this since it looks like a cert issue and not an issue with the SDK. Feel free to reopen if the problem persists after updating the certs.