Support multiple protocols: HTTP and gRPC (and more?)

[ghost]

Very cool project and impressive use of lots of new stuff.

It says it proxies tcp, so does that mean that i can proxy a GRPC client server interaction, web sockets and maybe even udp stuff like QUIC ?

[anxiousmodernman]

Only HTTP via this simple hack: https://github.com/anxiousmodernman/co-chair/blob/master/backend/forwarder.go#L160-L164

I'll look into more protocols. Selecting the binary ones dynamically would be an interesting challenge. I think HTTP2 should probably be the first target, since that should presumably support grpc as well.

I think I'll probably need to clean up the build first, though.

[ghost]

yes http2 will mean getting grpc.

the guys at the IPFS project have written a multiprotocol muxer i think. Might help. https://github.com/libp2p/go-stream-muxer

then if you need to run many grpc servers in one cmux seems to be the ticket. https://github.com/soheilhy/cmux

All giants to stand on top of !

On Thu, 3 May 2018 at 11:56 Coleman McFarland notifications@github.com wrote:

Only HTTP via this simple hack: https://github.com/anxiousmodernman/co-chair/blob/master/backend/forwarder.go#L160-L164

I'll look into more protocols. Selecting the binary ones dynamically would be an interesting challenge. I think HTTP2 should probably be the first target, since that should presumably support grpc as well.

I think I'll probably need to clean up the build first, though.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/anxiousmodernman/co-chair/issues/10#issuecomment-386245370, or mute the thread https://github.com/notifications/unsubscribe-auth/ATuCwqMNCj-8z3N_ifquGVzOtI6X8-vtks5tutRjgaJpZM4Tww_7 .

[anxiousmodernman]

After a cursory glance at cmux, it seems very promising. We will need to split HTTP variants from gRPC on a separate port/listener, but that isn't so bad. Thanks for the suggestion!

[anxiousmodernman]

Some additional notes:

gRPC over HTTP2 documented here: https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md#appendix-a---grpc-for-protobuf

I think creating and serializing a slice of cmux's lower-level Matcher type is what we want, rather than wrapping our listener with the Cmux listener. I'd like to support adding/removing backends without restarting the listener, and the Cmux interface doesn't (?) support that.

We already sniff some bytes into a buffered reader for plain old HTTP, and we can hand that buffer to a Matcher: https://godoc.org/github.com/soheilhy/cmux#Matcher

[anxiousmodernman]

Some notes on where I'm at with this:

• Getting closer to a working (if not 100% compliant) implementation of this. A unary grpc method seems to work, but I'll want to test streaming methods before merging to master. Like before, we lean heavily on tls.Config's GetCertificate method. When receiving a connection, we check the server name in the TLS "hello" and pick a cert from the database. This works the same for gRPC.
• authority in HTTP2 is the equivalent of Host: in HTTP1. I think we will continue to map both of these to Domain in the database, and pick a backend based on that. No rewrites or other header inspection yet.
• We are doing too many allocations right now, as well, and would benefit from buffer re-use.