Document gRPC and Protocol Buffer features that support HTTP transcoding

[timburks]

The ability to map gRPC services to REST-like APIs is a very useful feature and design pattern. It allows developers to focus on building a gRPC service that in many ways matches the procedural way their software already works, while still providing a path for users to use JSON/REST when they prefer it.

I would like to be able to promote this by pointing people to reference documentation that describes the mapping in a standard way. With a standard mapping, we can encourage multiple implementations and service developers can expect them to work consistently.

Currently one such mapping is provided by Cloud Endpoints. I'd like for that or something similar to become part of the gRPC documentation.

[LisaFC]

The standard mapping used by Endpoints (and indeed inside Google) is documented in more detail in the API Design Guide, which we should also at least link to - it already links back to grpc.io.

[LisaFC]

Would a good approach be to add a document called Integrating With Other Systems or similar, and then adding a HTTP/JSON Transcoding section (or just have a separate Transcoding concepts doc if this is substantial enough) that tells users:

• Some reasons why people do this/this is feasible (eg gRPC uses HTTP2 so your application can expose both REST/HTTP1.1 and RPC interfaces on the same port)
• Link to possible mechanisms for doing this, eg Endpoints, we have a blog post where a user describes creating their own proxy (http://www.grpc.io/blog/coreos), etc.
• Link to API Design Guide for advice on designing a transcoding-friendly service and some standard mappings

I don't want to end up reinventing the wheel and including our own mapping documentation (having been involved in the API Design Guide, those mapping docs involved many months of work by a lot of people, and they've been in heavy use inside Google for the last few years), and I don't think we want to end up documenting other people's transcoding tools/proxies.

[timburks]

It seems likely that there will be multiple transcoding implementations used within Google (if there aren't already) and third-party implementations also seem likely, so defining the mapping in an implementation-independent way would help maintain consistency, and that would benefit gRPC users.

I'd pose this as "Write a gRPC interface, get a free REST API" through standard mappings that are defined as part of gRPC. We've documented them in http.proto, and perhaps elevating the documentation there is the best way to implement this. (There are instances of google.api.http throughout the style guide, but I didn't see a place where the google.api.http option is particularly described - apart from http.proto).

The approach in the style guide generally goes from gRPC to REST through transcoding - it assumes that a developer is starting with a gRPC API description. To advocate this methodology, I also want to speak to people who primarily think in terms of REST and help them see gRPC as a better way to build their REST APIs. gRPC is like Django (and many other web frameworks), because it's mapping HTTP requests to functions, but unlike Django and nearly every other web framework, those functions can be distributed over a network and written in ten different languages. To put it bluntly, we've found that gRPC is the best way to build REST APIs and want to better share that experience.

[timburks]

Here's a draft that I wrote by starting with the comments in http.proto and trying to approach the point of view of REST API authors.

[LisaFC]

I like that, though I'm not sure about duplicating content from http.proto, which is already web-page-i-fied, eg, here: https://cloud.google.com/endpoints/docs/grpc-service-config/reference/rpc/google.api#google.api.Http. That's in the Endpoints docset but we could also add that generated content with some context to the end of the style guide/the APIs docset if the rest of the team are happy with that - that would work as elevating the documentation there?

Then we could write a more detailed conceptual doc aimed at REST users in the gRPC guide with a couple of examples, and then just point them at the detailed mapping resources.

[LisaFC]

Or we might be able to generate content from http.proto without the Cloudsite templating and put it in grpc.io, though that might be a bit fiddly.

[timburks]

Other structures for this would be fine, but after rereading my draft I don't think the draft goes far enough in empathizing with REST service creators. That explanation (and any other based on http.proto) starts with a gRPC service definition and derives an HTTP service. I think that if we started an in-person presentation to REST developers like that, they would leave immediately.

I think it would be better to start a simple example with a list of REST operations which we then follow with an explanation of how to implement each with a gRPC service. This is a completely different (backwards) approach to how we explain HTTP transcoding to internal developers, because internal developers are already committed to using protos, while external ones are probably just exploring.

So the document that I have in mind seems to diverge pretty far from the http.proto comments, which means that maybe it's fine to keep the more literal web export that you linked if we also write something directly aimed at REST service developers who are curious about gRPC. (IMO)

[LisaFC]

Catching up on old issues, Tim are you still interested in writing something specifically aimed at REST service users that we can combine with a http.proto reference doc?