Lazy evaluation of compression decision in compressor_filter for http response w/o content-length

[yuanzhe-ge]

If you are reporting any crash or any potential security issue, do not open an issue in this repo. Please report the issue via emailing envoy-security@googlegroups.com where the issue will be triaged appropriately.

Title: One line description Buffer response header in compressor_filter and perform compression decision when receiving first data frame (encode_data)

Description:

Describe the issue.

Snap leverages on compressor_filter to perform response compression for gRPC responses. gRPC server doesnt add content-length to response header that the compressor filter will compress small response payload as well , which causes some overhead in egress bandwidth.

What Snap does(already running in prod env) is to perform a lazy evaluation when receiving first data frame instead of buffering all data frames.

Snap want to seek some feedback from envoy community that

1. this operation is feasible (Snap can create a draft PR to review later)
2. upstream the change to envoy

[optional Relevant Links:]

Any extra documentation required to understand the issue.

[daixiang0]

Sounds great that avoid compression of small data, my concern is what is the "best" threshold for most cases then we can use it as default.

[yuanzhe-ge]

Snap has ABed multiple min threshold. for messaging service with 1024 bytes as min threshold, we are looking at 40% egress bandwidth saving with slight regression in P50(1-2ms) for e2e network latency.

[ggreenway]

cc @kbaichoo @mattklein123

[mattklein123]

Sure this sounds like a good idea!