search

grpc / grpc-java

The Java gRPC implementation. HTTP/2 based RPC
https://grpc.io/docs/languages/java/
Apache License 2.0
11.29k stars 3.8k forks source link

Test that newly-added Java 8 ALPN works #6997

Open ejona86 opened 4 years ago

ejona86 commented 4 years ago

In https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8230977 Java 8 received a backport of the Java 9 ALPN API. It appears to be available starting in 8u251 or u252. Support was added to Netty in https://github.com/netty/netty/pull/10196 and released in 4.1.49.

Since we do some of our own detection, we should make sure it works with these more recent Java 8 versions. Even though Java 8 has very poor AES GCM performance, if it is an additional option it can help users on nicher platforms why may not need high throughput. We will also need to update SECURITY.md.

It may make sense to split out a separate issue for OkHttp.

ignasi35 commented 4 years ago

hi @ejona86,

I've done a quick test and can reproduce a few test errors as soon as I use JDK8u252 (AdopOpenJDK).

Steps to reproduce

  1. install AdoptOpenJDK 8 u252 (e.g. using SDKMan: sdk use java 8.0.252.hs-adpt
  2. git clone master
  3. skip codegen and android via gradle.properties
  4. run ./gradlew :grpc-netty:test

The following tests fail: (copy/pasting from the report)

    ProtocolNegotiatorsTest. clientTlsHandler_userEventTriggeredSslEvent_supportedProtocolCustom
    ProtocolNegotiatorsTest. clientTlsHandler_userEventTriggeredSslEvent_supportedProtocolH2
    ProtocolNegotiatorsTest. clientTlsHandler_userEventTriggeredSslEvent_unsupportedProtocol
    ProtocolNegotiatorsTest. serverTlsHandler_userEventTriggeredSslEvent_supportedProtocolCustom
    ProtocolNegotiatorsTest. serverTlsHandler_userEventTriggeredSslEvent_unsupportedProtocolCustom
    ProtocolNegotiatorsTest. tlsHandler_userEventTriggeredSslEvent_supportedProtocolH2
    ProtocolNegotiatorsTest. tlsHandler_userEventTriggeredSslEvent_unsupportedProtocol
    TlsTest. basicClientServerIntegrationTest[JDK]
    TlsTest. clientRejectsUntrustedServerCert[JDK]
    TlsTest. noClientAuthFailure[JDK]
    TlsTest. serverRejectsUntrustedClientCert[JDK]
ignasi35 commented 4 years ago

Netty released a fix in 4.1.49.Final. As of this writing, 4.1.50.Final is already out.

I've tried bumping the dependency in the build definition to 4.1.50.Final but I must be missing something since there are still some test failures:

    ProtocolNegotiatorsTest. clientTlsHandler_userEventTriggeredSslEvent_supportedProtocolCustom
    ProtocolNegotiatorsTest. clientTlsHandler_userEventTriggeredSslEvent_supportedProtocolH2
    ProtocolNegotiatorsTest. clientTlsHandler_userEventTriggeredSslEvent_unsupportedProtocol
    ProtocolNegotiatorsTest. serverTlsHandler_userEventTriggeredSslEvent_supportedProtocolCustom
    ProtocolNegotiatorsTest. serverTlsHandler_userEventTriggeredSslEvent_unsupportedProtocolCustom
    ProtocolNegotiatorsTest. tlsHandler_userEventTriggeredSslEvent_supportedProtocolH2
    ProtocolNegotiatorsTest. tlsHandler_userEventTriggeredSslEvent_unsupportedProtocol
    TlsTest. basicClientServerIntegrationTest[JDK]
    TlsTest. clientRejectsUntrustedServerCert[JDK]
    TlsTest. noClientAuthFailure[JDK]
    TlsTest. serverRejectsUntrustedClientCert[JDK]
ejona86 commented 4 years ago

Make sure you also check to see if netty_tcnative in the build.gradle needs to be bumped as well.

ignasi35 commented 4 years ago

netty_tcnative is io.netty:netty-tcnative-boringssl-static:2.0.30.Final on master and that's the latest version already

ignasi35 commented 4 years ago

@ejona86 I've just realized I didn't bump the Jetty ALPN dependency. That fixes the issue. I've raised #7060

kiwi1969 commented 3 years ago

Today I grabbed latest libraries-bom v9.0.0 to see if gRPC pubSub is working yet on JDK8 without a tcnative.so

It uses grpc-netty-shaded 1.31.1, which then uses netty-tcnative-boringssl-static.version=2.0.30.Final (same as previously) which then uses io.netty/* version 4.1.48.Final (ie not the 4.1.49.Final that is required for ALPN)

I just want to update, that I am using IBM Z/OS, which only has JDK8 and no libtcnative.so. We have multi-million dollar cloud migration hinging on this method working.

The ALPN support from JDK9 was initially back ported by IBM into JDK 1.8.0_241 / JRE build 8.0.5.15 I tested with libraries-bom v9.0.0 with this JDK version, both with and without jetty ALPN agent, but get below errors in PubSub INFO: Java 9 ALPN API unavailable (this may be normal) INFO: netty-tcnative unavailable (this may be normal) Could not find TLS ALPN provider; no working netty-tcnative, Conscrypt, or Jetty NPN/ALPN available

The ALPN support above, was then replaced with the Oracle JDK9 code in JDK 1.8.0_251 /JRE build 8.0.6.11 I tested with libraries-bom v9.0.0 with this JDK version, with and without Jetty ALPN agent, but get below errors in PubSub INFO: Java 9 ALPN API unavailable (this may be normal) INFO: netty-tcnative unavailable (this may be normal) Could not find TLS ALPN provider; no working netty-tcnative, Conscrypt, or Jetty NPN/ALPN available

*Can you please let us know when you expect to have a version of the above with the newer io.netty/ modules ? Also, does the old io.netty 4.1.48.FINAL actually matter, if the Java9 ALPN is detected ?**

kiwi1969 commented 3 years ago

I see we have an upgrade to gRPC jars for v1.32.1, so io.grpc.netty.shaded should now use newer Netty (that should support ALPN on newer versions of JDK8). But when I try PubSub with the newer 1.32.1 jars each item in group(io.grpc), I still get failures.

What should the log look like, when gRPC detects the JDK8 back ported ALPN ? ie Is there any message like "INFO: Java 8 ALPN API available"

ejona86 commented 3 years ago

@kiwi1969, only SunJSSE and Conscrypt is supported with the Java 9 ALPN API. Supporting IBM's security provider would be a separate issue.

kiwi1969 commented 3 years ago

Oh, I see. You guys explicitly request "SunJSSE" in the code. So then the provider name mismatches with the IBM supplied "com.ibm.jsse2.IBMJSSEProvider2" I think I should be able to get it working simply by code to additionally try the IBM provider. (In addition to using the newest gRPC libraries instead of the ones mentioned in the google-cloud-bom)

I see someone has already done this work - any idea why this pull to support IBMJSEE2 wasn't implemented ? https://github.com/grpc/grpc-java/pull/5374

kiwi1969 commented 3 years ago

OK, I testing my own change to support IBMJSEE2, which is very similar to #5374 but superior coding. This worked perfectly for my on Z/OS running IBM JDK8 (IBM back ported the ALPN Java9 code some time ago)

I did have to specify the IBM specific JVM parm below, else we get TLSv1 issue with code trying to use blocked ciphers -Dcom.ibm.jsse2.overrideDefaultTLS=true

I created Pull request #7422

I guess this shows that the Java8 ALPN works (at least on Z/OS) - Do you want to close this issue ?

Note that as many Z/OS users would have hardware such as ZIIP processor card and Crypto Accelerator cards, the Java encryption workloads may be offloaded, and actually run faster than on other systems. ie Even though you state that Java8 has poor AES GSM performance, the IBMJSEE2 provider should take care of that.