gRPC-web protocol wrapper/proxy

[calvellido]

It just so happens that the gRPC protocol can't be directly used from a browser environment. Not because it doesn't exist a JavaScript implementation, but due to browser limitations. Being the main issue, among other setbacks, the lack of trailer metadata support on browsers / fetch implementations.

https://github.com/grpc/grpc/issues/2786 https://github.com/whatwg/fetch/issues/34#issuecomment-235318684

With this in mind, the approach taken by Google/gRPC team is to set an analog gRPC spec compatible with web browsers. This is needed until fetch/streams API spec is fully supported on web browsers, which can occur in ~2-3 years, then a browser could speak native gRPC protocol, being based on the JS node implementation.

They specifically indicate in the document that the protocol is designed to make it easy for a proxy to translate between the protocols as this is the most likely deployment model. The published protocol proposal can be consulted here:

https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-WEB.md

This document is essentially a delta of the differences with the protocol details specified in the official native gRPC wire protocol. Some of these needed changes include (but are not limited to):

• Specific Content-Type (application/grpc-web, application/grpc-web-text).
• Support for any HTTP/*, not only HTTP/2.
• Use of message framing, instead of HTTP/2 transport mapping (probably due to this last point, more on the differences between them here). And with this including the RPC response status as part of the response body, since on native gRPC this status is part of the trailer.
• Not setting the User-Agent explicitly, letting the browser to do that as expected, and use a custom header (X-User-Agent) that take the user agent native spec user agent role.

The message framing and the trailer considerations look like the meatier parts of this, and it doesn't look like something trivial. The fact that we are moving in a somehow greenfield territory makes it quite a challenge, but if any indication, the Improbable Go wrapper implementation can be used as insight.

The gRPC team also notices that besides the spec, its own implementation of grpc-web was initially developed for Google's own projects (...) and they are yet to go through the process to open-source the code base, but until that moment comes, we can rely on the Improbable spec interpretation, as it's proved to be working.

Another approaches worth to be discussed could be:

• Use Improbable Go gRPC-web proxy server.
• Use gRPC gateway, which translates a RESTful JSON API into gRPC or gRPC-websocket-proxy.
• Use a simplified custom wrapper that only operates under HTTP/2, and specifically coded to our needs? 🤔

Further info and relevant discussions on this topic:

https://improbable.io/games/blog/grpc-web-moving-past-restjson-towards-type-safe-web-apis https://github.com/grpc/grpc/issues/2786 https://github.com/grpc/grpc-experiments/issues/159 https://github.com/grpc/grpc/issues/8682#issuecomment-340079053

[vejeta]

It could be interesting, no matter the final solution, to create a gRPC-web proxy server in Scala, instead of Go.

[juanpedromoreno]

I'm starting working on this, thanks for such as detailed description @calvellido

[juanpedromoreno]

Initially, this would be an initial approach working together with the frees-rpc:

@service
trait MyService {

  @rpc(Avro)
  @http(GET, "/v1/service/unary")
  def unary(a: Request): FS[Response]

}

The only difference, when defining this protocol would be the @http annotation, which receives two parameters:

• HTTP method.
• REST Path where the http service will be available.

The HTTP method type ADT would be:

sealed trait MethodType extends Product with Serializable
case object GET         extends MethodType
case object POST        extends MethodType
case object PUT         extends MethodType
case object DELETE      extends MethodType
case object PATCH       extends MethodType

Ideally, the gateway server would be derived by the scalameta macro, and this service also registered to it automatically.