when will In-process proxies be ready?

[taojoe]

the java version in-process proxies has some issue, the request will do 3 times to grpc service, while the client does 1 time. and the java version match request to java code, java codes may have prefix packages, it's not transparent, it may need to change some code in the in-process proxies java code.

it there a plan to support in-process proxies offically?

[sampajano]

Thanks for asking!

Sorry unfortunately that the Java in-process proxy project is on hold right now and is not being actively supported at this moment.. (I'll make a documentation update to mention so..)

Apologize for any inconveniences!

[pbi-qfs]

@sampajano May you elaborate a bit on why you put the development of the java proxy on hold? Is it due to manpower or do you have other doubts?

In some scenarios (e.g. running a grpc-based electron app locally with embedded app server), you cannot use envy in a docker container (since obviously you do not want a user to install docker on his desktop pc just to start an otherwise self contained app) and go requires different binaries for different operating systems, so you loose the portability of your app. The Java proxy is your redemption here...

[sampajano]

@sampajano May you elaborate a bit on why you put the development of the java proxy on hold? Is it due to manpower or do you have other doubts?

No particular doubts that i'm aware of.. Mainly just prioritizations.. :)

Sorry for your inconveniences.. we encourage you to look for other ecosystem solutions too :)

[niloc132]

@pbi-qfs over at https://github.com/grpc/grpc-java/pull/8596 we have a grpc servlet that works with either javax.servlet or jakarta.servlet apis, and provides application/grpc support. I have since developed an application/grpc-web servlet filter that can stand in front of that grpc servlet, and transparently hand off all handling to it. This approach differs from the connector repo, which directly creates the java service by expected name (using reflection, avoiding ServiceDefinitions and interceptors), and as such might be more generally useful.

[wirekang]

@niloc132 Can you give more information about grpc-web servlet filter? Did it deployed?

[niloc132]

Sorry for the delay @wirekang - I hope the details in this writeup are helpful for you:

Yes, it works great. Unfortunately, the previous issue took a very long time to resolve, and we've basically reached a stable state... and stability is a terrible condition for "let's spend a bunch of time on this!" even to do a writeup.

If you (and this grpc/grpc-java's maintainers) would be amenable, open an issue for this in grpc-java and I'll see about contributing our work? that seems to be a more appropriate place than this repo.

Here's a summary of what we know, condensed to bullet points.

• We forked the early work from the above PR into our own repo, and labeled it as apache licensed, and we ship artifacts to maven central with each of our releases.
• gRPC-web's own in-browser client cannot handle streaming binary, a requirement of ours. We looked at the https://github.com/improbable-eng/grpc-web/ client instead, and found it capable of server-streaming binary payloads (though client-streaming still isn't an option, more on that later).
• gRPC doesn't work at all over http/1.1, and while gRPC-web pretends to work, streaming doesn't work beyond holding the http/1.1 connection open and just sending more messages (but preventing the browser from using that socket again) until http timeout hits.
• On the other hand, h2 doesn't work in browsers without SSL (i.e. there is no h2c for browsers), so an SSL certificate is required to use h2, and thus to use grpc-web.
• Unfortunately, there are many environments where it is difficult to deploy SSL for development or testing purposes (either through inexperience of users clicking through SSL warnings, or IT forbidding usage of invalid certificates, etc), so forcing SSL is not always an option. Thus h2 isn't an option.
• Fortunately, the https://github.com/improbable-eng/grpc-web/ repository above offers a grpc-web proxy written in Golang that makes gRPC streams available over websocket transports, and their JS client can also support that transport. And websockets work without SSL.
• Unfortunately, browsers still have a limitation about how many concurrent websockets can be running (per origin I believe?), so we made a slightly different, nearly compatible transport that is capable of using a single websocket to transport many concurrent streams. We implemented this using an alternative subprotocol to the one advertised by the above implementation (unfortunately a bug in the original implementation prevents it from negotiating subprotocols directly - I contributed a fix at https://github.com/improbable-eng/grpc-web/pull/1112, but it was never released). We do not at this time have a plain JS/TS implementation compatible with improbable-eng/grpc-web's browser client for this, but it should not be difficult to build one.

Our conclusions:

• io.grpc.servlet.jakarta.web.GrpcWebFilter (from io.deephaven:deephaven-grpc-servlet-jakarta) serves as an in-process proxy for supporting binary (text is not supported) grpc-web payloads. This can be installed in any servlet container already running the jakarta GrpcServlet, available either from the same artifact or from grpc-java itself. There's nothing to customize, it just works (and will support http/1.1 if your use case does). Note that we don't support javax.servlet, but there's no reason that this couldnt be added.
• io.grpc.servlet.web.websocket.WebSocketServerStream (from io.deephaven:deephaven-grpc-servlet-websocket-jakarta) is a jakarta.websocket.Endpoint implementation, offering a compatible implementation of the websocket mentioned above - but in-process rather than requiring a proxy. It isn't terribly easy to use - it isn't a "proxy" like the filter above, but instead serves as a replacement for the ServletAdapter from the original PR. Within our own project, we've modified the ServletAdapter type to let us instantiate this type (and the one in the next point), sharing the transport listener, attributes, etc.
• io.grpc.servlet.web.websocket.MultiplexedWebSocketServerStream uses the other subprotocol above, allowing a single websocket to roughly function like a single tcp socket for a set of h2 streams - the first connection requested in this way will be the URL that the client connects to - all future calls/streams will use the same websocket (i.e. the same URL), so each call will send its actual URL as part of the initial request. Both of these implementations can be enabled at the same time, a client will negotiate with whatever it can talk to.
• For production use cases, or deployments to servers with TLS certs, we use the GrpcWebFilter. For localhost development or other non-production use cases where TLS isn't available/enabled, we use the multiplexed websocket implementation. Both perform well, and while we have a few bugs to address some edge cases, they all seem to center around cases where the client has disconnected already, and the server doesn't handle it properly.

[jazdw]

@niloc132 I wish I had seen your comment earlier, I had no idea there was support for gRPC-Web in grpc-java. We modified the old Servlet code from this repo and made it conformant with the Connect conformance tests for gRPC-Web. https://github.com/RadixIoT/grpc-web-java

[niloc132]

Thanks for the link - going from the README, it appears that your project works a little differently than what I described, as yours functions as a proxy (default implementation is a separate gRPC client... via InProcessChannelBuilder) while the code I shared above is just translation so that the same Java gRPC server can offer both gRPC and gRPC-web without the need of a proxy (which greatly simplifies local development or simple deployments). Yours might give you some flexibility that we weren't concerned with right away, but I suspect will be somewhat heavier overall. I played with adding a proxy implementation, but never got far enough to be worth publishing anything (and it never was necessary for our purposes).

Feel free to contact me via email if you want to discuss further, my email is on my commits.

[jazdw]

Thanks for the link - going from the README, it appears that your project works a little differently than what I described, as yours functions as a proxy (default implementation is a separate gRPC client... via InProcessChannelBuilder)

Yep, indeed it does. It will allow you to configure routing services to different channels, which could be in-process or an external gRPC server via TCP/TLS.

while the code I shared above is just translation so that the same Java gRPC server can offer both gRPC and gRPC-web without the need of a proxy (which greatly simplifies local development or simple deployments).

As I understand it you would have to be running a gRPC server via a servlet instead of via Netty right?

Yours might give you some flexibility that we weren't concerned with right away, but I suspect will be somewhat heavier overall.

Yes most likely.

[niloc132]

As I understand it you would have to be running a gRPC server via a servlet instead of via Netty right?

Correct - we went down this path because the grpc-netty codebase makes it extremely difficult to add some other in-process grpc-web handler at all. We have found no appreciable difference for our use cases between the netty vs servlet (for which we are using Jetty 11) implementation in terms of gRPC performance, and using a servlet makes it not only possible to add grpc-web to the server, but also offer other static web resources.