chunked transfer encoding support needs improvement

[vagaerg]

The proxy currently supports chunked transfer encoding as an "alternative" to AWS chunked encoding, meaning that a request is only ever considered to be either using HTTP chunked transfer encoding or aws-chunked encoding.

The two are however fundamentally different: one of them is a transfer encoding while the other is a content encoding. It is possible to send AWS chunked data in an HTTP chunked stream, for instance.

AWS is unfortunately not very prescriptive as to what sort of HTTP chunked transfer encoding support they offer, some info is available here and here.

There are 2 main test cases where I wanted to see how Ceph and Minio compare to each other, then decide what the proxy should do:

1. "raw" HTTP chunked transfer encoding: standard (not aws-chunked) content transferred using HTTP chunked encoding
2. aws-chunked content transferred using HTTP chunked transfer encoding

For (1), there is an interesting corner case, particularly around how we should support unsigned payloads:

• It is unclear whether UNSIGNED-PAYLOAD should be considered acceptable when doing HTTP chunking, since the docs state "Use this when you are uploading the object as a single unsigned chunk."
• The other option is to use STREAMING-UNSIGNED-PAYLOAD-TRAILER, but trailer headers are tricky to work with and not widely supported (in fact, the script I linked above resorts to writing HTTP requests directly over a socket to support them, since most libraries do not)

Overall I ran the following test cases, showing Minio and Ceph results and the aws-proxy behaviour as of #131 :

Content	Content-Length	x-amz-decoded-content-length	x-amz-content-sha256	Transfer-Encoding	Content-Encoding	Trailer header	Minio result	Ceph result	aws-proxy
aws-chunked content	Set	Set	STREAMING-AWS4-HMAC-SHA256-PAYLOAD	none	aws-chunked	N/A	200	200	200
aws-chunked content	Set	Not set	STREAMING-AWS4-HMAC-SHA256-PAYLOAD	none	aws-chunked	N/A	400	400	411
http-chunked aws-chunked content	Not set	Set	STREAMING-AWS4-HMAC-SHA256-PAYLOAD	chunked	aws-chunked	None	200	200	200
http-chunked aws-chunked content	Not set	Set	STREAMING-AWS4-HMAC-SHA256-PAYLOAD	chunked	aws-chunked	x-amz-checksum-sha256	400	200	400
http-chunked aws-chunked content	Not set	Not set	STREAMING-AWS4-HMAC-SHA256-PAYLOAD	chunked	aws-chunked	None	411	400	411
http-chunked aws-chunked content	Not set	Not set	STREAMING-AWS4-HMAC-SHA256-PAYLOAD	chunked	aws-chunked	x-amz-checksum-sha256	411	400	411
http-chunked content	Not set	Set	UNSIGNED-PAYLOAD	chunked	not set	None	411	200	200
http-chunked content	Not set	Set	UNSIGNED-PAYLOAD	chunked	not set	x-amz-checksum-sha256	411	200	400
http-chunked content	Not set	Not set	UNSIGNED-PAYLOAD	chunked	not set	None	411	200	411
http-chunked content	Not set	Not set	UNSIGNED-PAYLOAD	chunked	not set	x-amz-checksum-sha256	411	200	411
http-chunked content	Not set	Set	STREAMING-UNSIGNED-PAYLOAD-TRAILER	chunked	not set	None	411	400	200
http-chunked content	Not set	Set	STREAMING-UNSIGNED-PAYLOAD-TRAILER	chunked	not set	x-amz-checksum-sha256	411	400	400
http-chunked content	Not set	Not set	STREAMING-UNSIGNED-PAYLOAD-TRAILER	chunked	not set	None	411	400	411
http-chunked content	Not set	Not set	STREAMING-UNSIGNED-PAYLOAD-TRAILER	chunked	not set	x-amz-checksum-sha256	411	400	411

As a tl;dr - as of my suggestion in #131 :

• The proxy now supports http chunking for standard content, including UNSIGNED-PAYLOAD (which Ceph allows but Minio doesn't)
• Trailer headers are ignored, so validation that relies on the inclusion of a trailer header is not supported - pretty much in line with Minio