When data is stored as (potentially paginated) Search Tables, not only can the JSON files be gzip compressed at rest, but they can be served as-is with a content-encoding: gzip header. This adds no burden to the web server, and minimal burden to the client receiving the data.
The vast majority of real-world HTTP clients understand content-encoding: gzip and transparently decompress data as it is received. However, according to the HTTP specification, we should only serve gzip-compressed data if the client requests it with an accept-encoding: gzip header.
If we specify that Search clients MUST be capable of dealing with content-encoding: gzip and MUST send an accept-encoding: gzip header, this will make it easy for tables-in-a-bucket Search implementations to serve compressed responses, speeding up transfers and reducing storage costs.
This issue was raised in a discussion on #98.
When data is stored as (potentially paginated) Search Tables, not only can the JSON files be gzip compressed at rest, but they can be served as-is with a
content-encoding: gzip
header. This adds no burden to the web server, and minimal burden to the client receiving the data.The vast majority of real-world HTTP clients understand
content-encoding: gzip
and transparently decompress data as it is received. However, according to the HTTP specification, we should only serve gzip-compressed data if the client requests it with anaccept-encoding: gzip
header.If we specify that Search clients MUST be capable of dealing with
content-encoding: gzip
and MUST send anaccept-encoding: gzip
header, this will make it easy for tables-in-a-bucket Search implementations to serve compressed responses, speeding up transfers and reducing storage costs.Notes on cloud support:
accept-encoding
: https://cri.dev/posts/2013-07-16-how-to-gzip-compression-of-css-and-js-files-on-s3-with-s3cmd/accept-encoding
in request: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/ServingCompressedFiles.htmlaccept-encoding
in request: https://cloud.google.com/storage/docs/transcodingaccept-encoding
: https://github.com/stefangordon/azure-storage-gzip-encoding/blob/master/README.md