Bidi streaming use cases for grpc-web

[wenbozhu]

Options, use cases to enable bidi streaming support with grpc-web

[arthur-tacca]

As I understand it you're interested in possible use cases for streaming? Here's one (but suggestions for better alternative implementations are welcome).

Use case: Returning pages in a multipage view (e.g. list of threads in forum software)

In this use case the front-end JS opens a stream and sends a request (e.g. GetThreads(forum="General", page=0)) and gets a response containing only one page of results. On the server side, a cursor into the database query results (e.g. using PostgreSQL) is kept open, and on future requests in the stream (e.g. GetThreads(forum="General", page=1)) this cursor is used to get the results. This ensures a consist view of data (e.g. you don't see the same thread on page 2 as you just saw on page 1). When the stream is closed, the database cursor is closed. Without streams you would need to add logic about when to keep open or close database cursors, and under load balancing you need to find a way to make sure requests from the same client end up at the same server.

[johanbrandhorst]

With the grpc-gateway wrapped in a websocket proxy (https://github.com/tmc/grpc-websocket-proxy) it's currently possible to do "bidi streaming" from the browser. Is using websockets for bidirectional streaming completely out of the question for grpc-web? I realise it would mean that any grpc-server would need to monitor for websocket connections, but maybe it could be a stepping stone before full trailer support is in the browsers?

[wenbozhu]

@johanbrandhorst

Thanks for the point.

We are in the process of launching grpc-web as an alpha release, and will publish the plan on bidi support (road-map) after Q2.

[HTChang]

@wenbozhu Do the grpc-web now support bidi streaming in the example? I saw the proto has defined bidi streaming API

  rpc FullDuplexEcho(stream EchoRequest) returns (stream EchoResponse);

but the doc of Known gaps / issues says

No client-streaming or bidi-streaming support

One of our use case would be in the search bar, customer will enter the keyword sequentially and the search request/response will be implemented via bidi streaming

[wenbozhu]

@HTChang no, the grpc-web repo version doesn't support bidi streaming. We probably should add a comment to the proto too to avoid confusion.

@stanley-cheung

[jonahbron]

@wenbozhu Have there been any developments on adding bidi support to the gRPC-Web spec?

[wenbozhu]

We don't yet have a future-proof transport story that allows us to provide cross-browser support and at same time serve both OSS and internal users. Will publish the road-map doc soon, and we are very close to making the repo public.

[jonahbron]

@wenbozhu I'm sure you've heard people scream "but websockets!" at you a million times. But I'm genuinely curious, can you explain or link to an article or something that explains why Websockets shouldn't be used here?

[johanbrandhorst]

https://github.com/grpc/grpc/issues/8682

[jonahbron]

Thanks @johanbrandhorst. Based on the comments on that issue (and the presence of a vote item in the ROADMAP.md), it sounds like it's still a possibility. Guess we'll just need to wait until the roadmap is updated with a decision.

[johanbrandhorst]

Indeed :). I'll be watching as eagerly as you are!

[wenbozhu]

Thanks for reading the ROADMAP.

This slide deck talked about websockets v.s. pure HTTP, with some stats from chrome.

We understand the desire to design a single API that works for both micro-service and Internet clients, among a few other use cases that could truly benefit from full-duplex streaming.

[DieMyst]

Hello, I want to write about my use case of bidi streaming. We are in the process of writing decentralized database. So, all data is encrypted and client must follow the b-tree with encrypted keys step-by-step to decrypt each key and say in which direction should be next step. We had good start in implementation this logic with grpc, but browser implementation without some wrappers (websocket, etc) of client is difficult now without bidi support. Our goal is the ability to use a decentralized database via browser having only a private key available.

[johanbrandhorst]

Note that Imorobables grpc-web implementation has a beta bidi implementation over websockets. https://github.com/improbable-eng/grpc-web.

On Wed, Mar 28, 2018, 20:37 Dima notifications@github.com wrote:

Hello, I want to write about my use case of bidi streaming. We are in the process of writing decentralized database. So, all data is encrypted and client must follow the b-tree with encrypted keys step-by-step to decrypt each key and say in which direction should be next step. We had good start in implementation this logic with grpc, but browser implementation without some wrappers (websocket, etc) of client is difficult now without bidi support. Our goal is the ability to use a decentralized database via browser having only a private key available.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/grpc/grpc-web/issues/24#issuecomment-377009614, or mute the thread https://github.com/notifications/unsubscribe-auth/AGTFd-ucgkhhsClxjZPA0ogZ-AcX6fwNks5ti-ZhgaJpZM4K6Cd6 .

[shanshanzhu]

@johanbrandhorst the improbable beta bidi version is really premature. It is not documented and has no example code. All it does is just exporting a transport function with websocket, but doesn't really wire it up with grpc. If I miss something (i.e. branch) there, could you help point me to the right place of code? Thanks!

https://github.com/improbable-eng/grpc-web/blob/709b592779c35e17f581f9bf866f4cac4b0b93fb/ts/src/transports/Transport.ts#L64

[johanbrandhorst]

@shanshanzhu the websocket proxy implementation is good. It wraps the Go http.ResponseWriter and io.Reader interfaces with websocket read/writes and passes the request to the gRPC Go implementation, so the wire protocol is pure gRPC-over-http2-over-websockets. Take a look at the implementation, it's beautiful :heart_eyes: https://github.com/improbable-eng/grpc-web/blob/72eb701d6f320ca324b3347c7925a720b553eae5/go/grpcweb/wrapper.go#L133, https://github.com/improbable-eng/grpc-web/blob/709b592779c35e17f581f9bf866f4cac4b0b93fb/ts/src/transports/websocket.ts#L51.

Furthermore, as I already pointed out in https://github.com/improbable-eng/grpc-web/issues/205, you can look at the tests for an example of how to use it: https://github.com/improbable-eng/grpc-web/blob/master/test/ts/src/client.websocket.spec.ts#L77.

The GopherJS gRPC-Web bindings have been using the websocket proxy on https://grpcweb.jbrandhorst.com since the start of the year.

Having said all of this, I think it is probably right to wait for Fetch API and Streams API to land in browsers before implementing this in this client.

[wenbozhu]

Re: websockets, we would welcome all the gateways that provide inter-operability with gRPC end-points.

WS over HTTP/2 is being finalized and will be available in Chrome. We will evaluate if it makes sense to make WS a transport for grpc-web (but the bar to deploy this for Google's web apps will be high) and there will be details such as ping-pong, half-close etc to spec out ...

[nhooyr]

@johanbrandhorst does this mean there is a client side grpc-http2 implementation thats reading everything from the web sockets?

[nhooyr]

@wenbozhu My use case for bidirectional streaming is for an online game where the client is sending message based on what the user is doing and the server is sending messages to clients to broadcast all events. I could just do this over plain web sockets with protobuf but it'd definitely be nice to see support for this. Can we implement something similar to what @johanbrandhorst noted in improbable's grpc-web repo? It seems like it'd be really simple given its just a wrapper around the protocol and we could document that bidirectional streaming is done over web sockets. Later on, with WS over HTTP/2, we could enable support for that and then later for the streams API. This approach would ensure broad browser compatibility.

[johanbrandhorst]

I don't think we should implement this in this repo until we have request streaming in Fetch. There is a third party, non-spec implementation that users can use if they need bidi streaming before then.

[nhooyr]

@johanbrandhorst the issue with that is Fetch will only work on bleeding edge browsers. I think its a pragmatic idea to include WebSockets over HTTP/1.1 as a fallback.

[johanbrandhorst]

I disagree. This will by necessity be bleeding edge tech. The spec talks about eventually speaking the HTTP2 protocol, which is not supported by todays browsers. I don't think the aim of this project is to provide backwards compatibility to old browsers.

[wenbozhu]

grpc-web does intend to support important browser use cases, i.e. features, versions, environments etc ... and the current spec is implementation details but important.

I don't necessarily agree tunneling grpc/http2 over websockets is a good or sane idea, and you are going to run into issues such as flow-control, half-close etc, to name a few.

One of the problems with websockets is that connectivity is not guaranteed (e.g. to support gmail) .. which means you always need a fallback. We don't yet want this level of complexity for grpc-web.

https://github.com/HTTPWorkshop/workshop2017/blob/master/talks/websockets.pdf

===

This bug is intended to collect use cases (title updated :) ... e.g. the case @nhooyr mentioned, could the API be actually better (e.g. resilient to failures) if client uses unary calls to post and uses server-stream to receive messages?

[johanbrandhorst]

@wenbozhu 403 on that document Im afraid

[wenbozhu]

fixed it.

[nhooyr]

I don't necessarily agree tunneling grpc/http2 over websockets is a good or sane idea, and you are going to run into issues such as flow-control, half-close etc, to name a few.

Couldn't flow control be implemented at the grpc-web level? Likewise with half close?

One of the problems with websockets is that connectivity is not guaranteed (e.g. to support gmail) .. which means you always need a fallback. We don't yet want this level of complexity for grpc-web.

In what way is the connectivity of WebSockets not guaranteed?

e.g. the case @nhooyr mentioned, could the API be actually better (e.g. resilient to failures) if client uses unary calls to post and uses server-stream to receive messages?

The issue with that would be the constant authentication checks on every single message (of which there would be a lot) when doing an RPC. A bidirectional stream completely avoids this by only authenticating the connection at the start. I guess I could give each client a token before they start a game and then they pass that token on each request and we just directly compare that token instead of hashing for performance reasons on each RPC but still its more complicated than just a bidi stream.

Furthermore, while unary RPC+server stream avoids depending on the client side streaming connection remaining open, its not really more resilient because it'd be very easy to just reopen the bidi stream and we still have to worry about the server stream closing anyway. I don't think its more resilient in my case.

[wenbozhu]

@nhooyr

Agreed, that performance win is a good case for bidi streaming.

Re: websocket connection failures, please check the doc on chrome stats. We don't really understand all the root causes, e.g. virus detection software etc.

Re: wire semantics, yes, grpc-web specific concepts need be introduced than just tunneling grpc/http2 over websockets (as a TCP-like bytestream). This to me adds unnecessary complexities than doing a grpc-web specific mapping to websockets (as a transport).

[nhooyr]

Re: websocket connection failures, please check the doc on chrome stats. We don't really understand all the root causes, e.g. virus detection software etc.

If that happens, there should be no fallback. Just throw an error. Its unfortunate but welp. I think in the vast majority of cases, web sockets will work fine and be a great fallback later for HTTP/2 WebSockets.

Re: wire semantics, yes, grpc-web specific concepts need be introduced than just tunneling grpc/http2 over websockets (as a TCP-like bytestream). This to me adds unnecessary complexities than doing a grpc-web specific mapping to websockets (as a transport).

Wouldn't grpc/http2 over web sockets take care of half close and flow control?

[wenbozhu]

grpc/http2 only passes the payload over websockets, not http/2 itself .. but if the latter is the case, I really don't know how to reason about this.

In any case, I'd agree that ws over http2 will make ws a viable transport for grpc-web (only).

[nhooyr]

grpc/http2 only passes the payload over websockets, not http/2 itself .. but if the latter is the case, I really don't know how to reason about this.

Ah that sounds fine to me. I see what you mean now. We'd need a small wrapper protocol around websockets as well.

[alethenorio]

Just passing by to add a use case for client streaming.

I was looking to be able to support an API for file upload. From what I can gather, uploading a (maybe large) file would require client streaming.

I looked further into adding an HTTP endpoint to the gRPC server in my Go application but it seems that the http endpoint supported by the gRPC Go library requires an http/2 connection and TLS (which is not ideal if you are fronting your app with a load balancer doing TLS termination) so I might as well write a REST endpoin instead of gRPC.

I am curious as to why this specific use case was not mentioned, maybe I missed another way of doing that?

[johanbrandhorst]

AFAIK file uploads in the browser are most appropriately handled with multi-part upload requests, a well established web standard. Furthermore, I don't think there's a clear mapping between it and gRPC methods, unfortunately. An interesting thought would be if the gRPC-Web proxy could automatically translate such requests to a client streaming request of some well defined format.

[wenbozhu]

+1 on what @johanbrandhorst said .... We have no plan (or mechanism) to support random media tunneling as part of grpc-web, but this could be a "grpc gateway" feature for developers who only want to deal with grpc end-points.

[leonfancy]

I'm working on a speech regonition project, browser records and streams audio binary data to gRPC server and server streams the intermediate result to client.

rpc recognize(stream Audio) returns (stream Text) {}

In this case, bidi streaming is essential.

[imans777]

@leonfancy Same here

[micfan]

somewhere docs write as this...

This feature finally will be implemented, then gRPC can own half of the share in the Browser side, just like what Node.js to JavaScript. No HTTP1 proxy, no WebSocket bridge, seamlessly combaine front and backend, no RESTful any more. Use cases are not important, what you want, what we have

[PavelSosin]

For those who adopt already released LSP and DAP protocols non-bidirectional gRPC without both client and server push features doesn't make any sense. Both protocols are full duplex protocols built on the top of WebSocket and JSONRPC. Even. If server is implemented as Web/NodeWorkers with near zero communication protocol latency transfer time for very big objects or long stream shifts end-user latency above allowed. It is not a rare case when the final object is created by parallel threads or reactive streams. Wise developer can always spread work among workers keeping single communication channel. But the result serialization and transfer time is critical! Compression of transfer buffer using plane JSZIP gave me more than 10 folds improvement in communication time between browser/nodejs main thread and worker. I expect the same and, even higher, improvement from gRpc using compressed protobuf. Unfortunately buffer compression object is available in Nodejs 10+ but not in Chrome and Edge browsers built on the same V8 engines.

[emmveqz]

@micfan when? where is it (going to be)? I need it!!

This feature finally will be implemented

[micfan]

@emmveqz These text is cited from some docs. I have turn to the raw gRPC. If there no so many requests for this feature, can push it off. Or, ProtoBuffer over WebSocket is OK.

[wenbozhu]

@leonfancy Yes, speech recognition is a canonical use case.

We will publish a detailed plan soon (by Nov.), i.e. what choices we have, their limitations, timelines, as well as considerations of future Web transports. E.g. it could be the case that we decide to ship a basic websocket based solution to unblock the development environment first if this is what everyone wants.

[hochhaus]

@wenbozhu It isn't the end of nov yet, but I am very excited about the bidi grpc possibility. Any update here?

[wenbozhu]

Sorry about the delay. Will get something published before the year end.

[wenbozhu]

Not the exact plan yet but please take a look: Road-map feedback Thanks!

[coryvirok]

Bump. :)

Am I correct in assuming that this functionality is blocked by Fetch full/half-duplex support? I'd be in favor of getting a working solution now and swapping it out for the more correct solution later. Considering this issue has been open since 2016, I'd say bleeding edge browser support is also OK for now since it will unblock many of us.

Thanks for all of the hard work on this.

[wenbozhu]

We are in active discussion with chrome/blink on the future streaming Web APIs, e.g. request-streaming via fetch, web-transport/quic. At this point, I feel any effort on creating a formal websocket based solution is not a good investment. I will post a "road-map in progress" shortly. Thanks.

[wenbozhu]

A quick update:

1. fetch request-streaming: we will help run some experiments with Chrome to address certain compatibility concerns. We hope that most browsers will add fetch request-streaming support soon.
2. web-transport: the IETF and Web API standards may take some time to finalize but Chrome will have early support (over quic).

With the development of the above new Web APIs, our current plan is not to invest on any websocket-based solution to minimize the complexity of grpc-web.

Will keep you updated on the progresses we are making.

[glerchundi]

@wenbozhu thanks for the update. It helps keeping hope on this issue and I second your strategy on going through QUIC instead of WebSockets.

[coryvirok]

I realize these types of features can take a while to implement, but it would be helpful to get a ballpark timeline for this. Is this likely going to take another month/6mo/year/more to implement? Is there anything the community can help out with to move this along?

[ydesgagn]

I would be willing to allocate 1-2 engineers on this to create a PR. Any guidelines we should follow so the PR can be accepted smoothly?

[jsmouret]

what about a webrtc data channel instead of websocket?