Configurable backend response decompression

Is your feature request related to a problem? Please describe.

Skipper proxy component uses http.Transport to perform backend requests. When Skipper client does not specify that it accepts gzip response encoding, the http.Transport adds Accept-Encoding: gzip request header and if backend serves compressed response the Transport will transparently decompress it before serving back to the client. This behavior can be configured by the DisableCompression http.Transport option:

    // DisableCompression, if true, prevents the Transport from
    // requesting compression with an "Accept-Encoding: gzip"
    // request header when the Request contains no existing
    // Accept-Encoding value. If the Transport requests gzip on
    // its own and gets a gzipped response, it's transparently
    // decoded in the Response.Body. However, if the user
    // explicitly requested gzip it is not automatically
    // uncompressed.
    DisableCompression bool

E.g. consider the following two routes:

bin/skipper -inline-routes='
backend: Path("/backend") -> logHeader("request") -> inlineContent("backend response\n") -> <shunt>;
test: Path("/test") -> setPath("/backend") -> "http://localhost:9090"'

Logs of GET request to the /test:

~$ curl -v localhost:9090/test 
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9090 (#0)
> GET /test HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.58.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Length: 17
< Content-Type: text/plain; charset=utf-8
< Date: Wed, 10 May 2023 13:38:49 GMT
< Server: Skipper
< 
backend response
* Connection #0 to host localhost left intact

show that request to the "backend" receives Accept-Encoding: gzip header added by the transport:

[APP]INFO[0003] GET /backend HTTP/1.1
Host: localhost:9090
User-Agent: curl/7.58.0
Accept: */*
Accept-Encoding: gzip

This behavior means that gzip-aware backend and Skipper spend time compressing and decompressing response even when Skipper client did not ask for any compression.

Describe the solution you would like

Make proxy transport DisableCompression option configurable. Consider disabling transparent compression by default (this is potentially breaking change as it may cause changes in the system behavior due to increased skipper<->backend network consumption).

Describe alternatives you've considered (optional)

disable compression unconditionally
leave everything as is

Additional context (optional)

http.Response contains Uncompressed field that hints if response was transparenmty uncompressed by the transport:

    // Uncompressed reports whether the response was sent compressed but
    // was decompressed by the http package. When true, reading from
    // Body yields the uncompressed content instead of the compressed
    // content actually set from the server, ContentLength is set to -1,
    // and the "Content-Length" and "Content-Encoding" fields are deleted
    // from the responseHeader. To get the original response from
    // the server, set Transport.DisableCompression to true.
    Uncompressed bool

This data might be used to make a decision about enabling/disabling transport compression.

Would you like to work on it?

Yes

zalando / skipper

Configurable backend response decompression #2307