Accept-Encoding spec guidance for listing catalogs

[bmelville]

The /v2/catalog endpoint can come with an incredibly large response payload, mostly due to schemas. Since it is text and is capable of being compressed quite effectively, it would be beneficial to allow for encoded responses. We've seen some of our large catalog responses compressible from 124KB to 7KB using gzip.

Because this is a negotiation between HTTP client and server, it might also be worthwhile to provide guidance or requirements in the spec, in the form of what encoding formats are officially supported. This would allow brokers to implement a specific encoding, and would allow platforms to know what encoding to allow if available.

Here is the list of official supported encodings for HTTP.

Here is a good example of negotiation.

Allowing brokers to support this would not require them to, as an HTTP service is not obligated to act on an Accept-Encoding header.

Alternatively, we could choose to make this specifically not part of the spec (it's already part of HTTP in general), in which case brokers can choose to implement any encoding and platforms can choose to ask for any or all with no guidance from us.

[duglin]

I don't see any issue with adding a line to the spec to make it clear that people are free to do this even though its out of scope for us. I would think this would apply to all of the APIs and not just the catalog - especially the schema endpoint we're thinking about adding.

[mattmcneeney]

Closing due to inactivity, but please reopen if this is still a problem for anything.