interop test cases for gRPC compression

[hawkw]

• [ ] client_compressed_unary
• [ ] server_compressed_unary
• [ ] client_compressed_streaming
• [ ] server_compressed_streaming

[hawkw]

Blocked on implementation of gRPC compression.

[per-gron]

Notes on implementing gRPC compression:

It seems like deflate and gzip are the two main algorithms that need to be supported for basic compression support. flate2 seems like a suitable library to depend on for these formats.

The spec for gRPC compression is here.

The settings that can be provided for compression are: Compression algorithm (None, Deflate, Gzip) and compression level (None, Low, Medium, High). Compression should be configurable both "at channel creation time" (I interpret this as Tower Service creation time) and "at response time", ie per RPC.

I'm unsure of what the concrete API should look like. Looking at the helloworld client example it seems like "channel creation time" settings can be provided like Greeter::new_with_config(conn, Config { compression_algorithm: Deflate, compression_level: Medium, ..Default::default() }). Per-RPC settings could be provided either by adding settings to the Request type, or by adding an extra config parameter to the RPC method say_hello in this example.

Similarly, on the server side, it can be provided when creating the service (server::GreeterServer::new_with_config) and in the Response object. I don't see a natural way to provide this configuration outside of the Response object on the server side: One option could be to provide some kind of Context parameter to the say_hello RPC handler method that allows setting compression settings (this is how the sync C++ API works), but I don't really see how that can work in an async environment.

It seems like adding the config options to the Request and Response types could make sense. This would also let the receiving peer see what the compression was, not sure if that is important though.

@seanmonstar @carllerche what do you think?

[per-gron]

More random thoughts (spoiler alert: they didn't lead anywhere really):

Compression is fairly similar to timeout; both are things that should have both a global setting and per-RPC overrides. It seems like the Tower/Finagle way of doing timeouts is via middleware (Tower)/filter (Finagle).

Middleware are easy to use as a global setting, but I guess they could fairly easily also be used for local overrides, as in

Timeout::new(client, Duration::from_millis(100)).say_hello(Request::new(HelloRequest {
    name: "What is in a name?".to_string(),
}))

(not entirely sure about the lifetimes involved here)

For timeouts this is nice but it's not quite directly applicable to compression, because compression matters also for responses. It's not clear to me what a middleware-style compression API would look like that allows servers to set compression of responses on a per-RPC basis.

[carllerche]

Thanks for the thoughtful write up. I agree with you in that compression should be a middleware. I hacked together a deflate middleware here.

I'm not sure the best way to make it all fit together though. It seems like compression is built into the gRPC protocol, so the middleware should probably be included by default in the grpc stack. Also, since compression is part of gRPC, one way to signal per RPC call compression would be to have a compressed flag on grpc::Request.

Thoughts?