search

storj / drpc

drpc is a lightweight, drop-in replacement for gRPC
MIT License
1.49k stars 49 forks source link

Document the protocol in explicit detail #7

Closed zeebo closed 3 years ago

zeebo commented 3 years ago

Some things that could use explicit documentation if one were to write a second, compatible implementation:

A good test would be to have someone other than those who have already worked on the code try to implement it in another language using only the documentation.

mjpitz commented 3 years ago

Is there a good way to help with some of this documentation? I've done this for a few protocols I've worked with in the past and wouldn't mind helping out with some of this. Once there's some documentation, I wouldn't mind taking a look at putting together a NodeJS implementation. Let me know how I can help out here.

zeebo commented 3 years ago

Awesome! So, I've enabled the wiki feature and created some pages that I'm going to be filling in. If you'd like to help, it should be the case that it's editable by anyone and some of the pages have links to the Go implementations the docs would be based on. I don't really know what counts as being good documentation for these sorts of things, so I'd even find outlines to be helpful.

I'm really excited to hear that you'd be willing to help out with a NodeJS implementation. One of the project goals is to be compatible with the protoc generated APIs and semantics, so I expect that's where most of the challenge lies. As a first step, I made wiki pages to capture some sample outputs for Go and NodeJS, and I'll be filling out the Go one shortly.

A lot of this is open ended at the moment, and that's because I don't really know what I'm doing. So, feel free to ask any questions and maybe we can figure something out.

mjpitz commented 3 years ago

I'll take a pass over the wiki pages sometime today or tomorrow. I'm conducting a handful of interviews throughout the course of the next couple of days and should have some time to sit down and read over them.

I've done a handful of protoc plugin projects here and there so I've got some context. I agree that the NodeJS implementation will be interesting. There are a few things that I think are going to be a challenge. For one, if we add WebSocket support for the browser, then there would need to be WebSocket server channel support across languages rather than just building on top of TCP. I think writing a TCP implementation with good interfaces would be a good start. That way, it should be easy to support a WebSocket transport. Not sure if y'all feel strongly about typescript vs more modern Javascript practices (I mostly work in typescript). I'll need to dive more into the Go side of the world, but I'm guessing supporting alternative transports wouldn't be that hard to do.

I'll definitely follow up with questions. Thanks for the starting point references!

zeebo commented 3 years ago

It looks like there has been some work done (and I just realized the name was the same lol) in a nodejs port (in typescript) in https://github.com/storj/drpc/issues/6 so I think we should try to coordinate all the nodejs stuff there going forward, and keep this issue about documentation. I'll make a wiki page to own a checklist of docs to do still (edit: https://github.com/storj/drpc/wiki/Docs:-TODO-LIST).

For websockets, that was brought up in https://github.com/storj/drpc/issues/5 and I did some weekend hacking to see how hard it would be, and it seems doable.

mjpitz commented 3 years ago

@zeebo let's move all JS discussion to #6. So far the wiki has been helpful for verifying my understanding of the go code. Once I'm done with the drpcwire package in node, I'll take another pass over the wiki.

zeebo commented 3 years ago

Calling this closed. Docs are a living deliverable, so it would just stay open forever otherwise. It also seems like a good portion of it is now documented as desired.