Support for TLS/plaintext Port Unification

[akh1lt]

Is your feature request related to a problem?

While trying to work with TLS, I was scanning through the current server setup. The NettyServerBuilder configures the port to be TLS or plaintext based on the presence or absence of SslContext, correspondingly. https://github.com/grpc/grpc-java/blob/012dbaf5be3fb0d532d977d288a0e42a58f30a7c/netty/src/main/java/io/grpc/netty/NettyServerBuilder.java#L352-L364

Resulting in a very little flexibility for the users to customize the port. Specifically, I was trying to see if we can accept both TLS & non-TLS connections on the same port without the need for creating a duplicate port. Netty demonstrates this through the doc.

Describe the solution you'd like

Approach1 Explicit public interface in NettyServerBuilder for enabling/disabling/multiplexing TLS. We could follow a similar approach of spiffing the initial bytes & dynamically configuring TLS.

Approach2 A public interface for adding child handlers to customize the connections. https://github.com/grpc/grpc-java/blob/012dbaf5be3fb0d532d977d288a0e42a58f30a7c/netty/src/main/java/io/grpc/netty/NettyServer.java#L228

Additional context

Relevant thread in stackoverflow: https://stackoverflow.com/questions/71484231/port-unification-in-grpc-java

[ejona86]

The code to encapsulate such an approach would be a ProtocolNegotiator. You'd need to make a ProtocolNegotiator (probably placing it in ProtocolNegotiators), and then expose it via a new *ServerCredentials class. NettySslContextServerCredentials would be an example of what the public-facing API would look like.

I need to discuss with other languages how they'd feel about Java supporting this feature; when one language supports a feature it puts pressure on other languages to support it as well.

That said, in Java we care more about such code being in our repo because the API is very unstable. In Go you can make this sort of thing yourself using the public API, I believe. So it may make some sense for Java to support it directly and other languages may just let "you do it yourself."

[ejona86]

CC @sanjaypujare, since we've talked about TLS+plaintext on a single port before

[sanjaypujare]

CC @sanjaypujare, since we've talked about TLS+plaintext on a single port before

Yes. One thing to mention is that for xDS managed servers this support is there (but not tested). For an xDS managed server the transport_socket field contains the TLS config info (which could specify plaintext) and this is done based filter_chain_match matching. So one can set up a filter chain where the some connections get configured with TLS and others get configured as plain-text - for example based on source IP addresses.

[ejona86]

One thing to mention is that for xDS managed servers this support is there (but not tested).

Not quite what is being discussed here. Here we want to support a client that may be TLS or plaintext. That requires sniffing the initial bytes. The support in xDS does indeed allow TLS+plaintext on a single port, but you have to make rules like based on the IP address. It isn't automatic. xDS itself supports the automatic approach (via tls_inspector+transport_protocol), but we don't support it.

[sanjaypujare]

... Not quite what is being discussed here. Here we want to support a client that may be TLS or plaintext. That requires sniffing the initial bytes. The support in xDS does indeed allow TLS+plaintext on a single port, but you have to make rules like based on the IP address. It isn't automatic. xDS itself supports the automatic approach (via tls_inspector+transport_protocol), but we don't support it.

That's correct. The dynamic config based on initial sniffing of bytes is not currently there - only rule based.

[ejona86]

Spoke with C and Go folks. Go has no concern, because it can indeed be done with the existing public API. C would need a new credentials type, similar to Java. But not much heart-ache about it if it becomes necessary.

@akh1lt, would you be willing to prepare a PR for this?

I expect there will be some bikeshedding for what name to call this credential. We have a API review meeting every-other-week (next one on the 31st) where we can resolve that question. As a working name, we can use io.grpc.netty.InsecureAndTlsServerCredentials.

[akh1lt]

@ejona86 Currently working on a local POC to add a custom ProtocolNegotiator using the InternalProtocolNegotiator.ProtocolNegotiator interface.

And multiplexing the handlers: InternalProtocolNegotiators.serverPlaintext().newHandler(grpcHttp2ConnectionHandler) InternalProtocolNegotiators.serverTls(sslContext).newHandler(grpcHttp2ConnectionHandler)

I'd be happy to raise a PR once my experiment goes through.

[Srinivas-Karre]

@akh1lt did you manage to get any workaround for this problem? We are also ran into the same kind of problem and looking for port unification for TLS and plaintext.

[akh1lt]

Yes, I was able to get it working. The basic idea remains the same

1. Create a custom protocol negotiator, something like this

   public class OptionalSSLProtocolNegotiator implements InternalProtocolNegotiator.ProtocolNegotiator {
   private SslContext sslContext;
   
   public OptionalSSLProtocolNegotiator(SslContext sslContext) {
   this.sslContext = sslContext;
   }
   
   public OptionalSSLProtocolNegotiator() {
   this.sslContext = null;
   }
   
   @Override
   public AsciiString scheme() {
   return AsciiString.of("https");
   }
   
   @Override
   public ChannelHandler newHandler(GrpcHttp2ConnectionHandler grpcHttp2ConnectionHandler) {
   ChannelHandler plaintext =
       InternalProtocolNegotiators.serverPlaintext().newHandler(grpcHttp2ConnectionHandler);
   ChannelHandler ssl =
       InternalProtocolNegotiators.serverTls(sslContext).newHandler(grpcHttp2ConnectionHandler);
   ChannelHandler decoder = new PortUnificationServerHandler(ssl, plaintext);
   return decoder;
   }
   
   @Override
   public void close() {}
   
   ProtocolNegotiationEvent getDefPNE() {
   ProtocolNegotiationEvent protocolNegotiationEvent = null;
   try {
     Field DEFAULT = ProtocolNegotiationEvent.class.getDeclaredField("DEFAULT");
     DEFAULT.setAccessible(true);
     return (ProtocolNegotiationEvent) DEFAULT.get(protocolNegotiationEvent);
   } catch (Exception e) {
     e.printStackTrace();
   }
   return protocolNegotiationEvent;
   }
   
   public class PortUnificationServerHandler extends ByteToMessageDecoder {
   private ProtocolNegotiationEvent pne;
   private final ChannelHandler ssl;
   private final ChannelHandler plaintext;
   public PortUnificationServerHandler(ChannelHandler ssl, ChannelHandler plaintext) {
     this.ssl = ssl;
     this.plaintext = plaintext;
     this.pne = getDefPNE();
   }
   
   private boolean isSsl(ByteBuf buf) {
     return SslHandler.isEncrypted(buf);
   }
   
   @Override
   protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out)
       throws Exception {
     if (in.readableBytes() < 5) {
       return;
     }
     if (isSsl(in)) {
       ctx.pipeline().addAfter(ctx.name(), (String) null, this.ssl);
       ctx.fireUserEventTriggered(pne);
       ctx.pipeline().remove(this);
     } else {
       ctx.pipeline().addAfter(ctx.name(), (String) null, this.plaintext);
       ctx.fireUserEventTriggered(pne);
       ctx.pipeline().remove(this);
     }
   }
   }
   }

2. Use it in the netty builder

   NettyServerBuilder.protocolNegotiator(new OptionalSSLProtocolNegotiator(sslContext));

[He-Pin]

I also want to explore this ProtocolNegotiator as we want to support gRPC and plain HTTP1.1 on the same port.