If a test makes use of a fixture server where the test data/file being served is already compressed the client will erroneously decompress the file. The following describes why.
On the client side, the aiohttp Client will automatically decompress response bodies when the Content-Encoding is one of the compressed types, like gzip.
On the server side the aiohttp Server will set Content-Encoding based on mimetype, which for gzip'd files with be 'gzip'.
--web_fileresponse.py:170--
if hdrs.CONTENT_TYPE not in self.headers:
ct, encoding = mimetypes.guess_type(str(filepath))
This will happen regardless of Accept-Encoding in the request header (which might be considered a bug in aiohttp). As you can see in the above code snippet one way to avoid this automatic setting of Content-Encoding is to set the Content-Type in the header, which is what we do here.
With this change in place the Accept-Encoding appears to be respected, presenting the client with the expected outcomes based on the various RFCs. Which ultimately results in a compressed file not being deflated by the aiohttp client. NOTE the Content-Length, Content-Type and Content-Encoding in the following data grabbed from a scenario served by pulp-smash.
Note that uncompressed data (Packages) will be compressed during transmission when Accept-Encoding is 'gzip'. This change should not affect tests based off pulp-smash, but if they do break it is more likely the test was working around this issue.
fixes #1309
If a test makes use of a fixture server where the test data/file being served is already compressed the client will erroneously decompress the file. The following describes why.
On the client side, the aiohttp Client will automatically decompress response bodies when the Content-Encoding is one of the compressed types, like gzip.
On the server side the aiohttp Server will set Content-Encoding based on mimetype, which for gzip'd files with be 'gzip'.
--web_fileresponse.py:170-- if hdrs.CONTENT_TYPE not in self.headers: ct, encoding = mimetypes.guess_type(str(filepath))
This will happen regardless of Accept-Encoding in the request header (which might be considered a bug in aiohttp). As you can see in the above code snippet one way to avoid this automatic setting of Content-Encoding is to set the Content-Type in the header, which is what we do here.
With this change in place the Accept-Encoding appears to be respected, presenting the client with the expected outcomes based on the various RFCs. Which ultimately results in a compressed file not being deflated by the aiohttp client. NOTE the Content-Length, Content-Type and Content-Encoding in the following data grabbed from a scenario served by pulp-smash.
-- curl -I -H 'Accept-Encoding: gzip' http://127.0.0.1:$PORT/debian/dists/nosuite/asgard/binary-ppc64/Packages.gz \ && curl -I -H 'Accept-Encoding: identity' http://127.0.0.1:$PORT/debian/dists/nosuite/asgard/binary-ppc64/Packages.gz \ && curl -I -H 'Accept-Encoding: gzip' http://127.0.0.1:$PORT/debian/dists/nosuite/asgard/binary-ppc64/Packages \ && curl -I -H 'Accept-Encoding: identity' http://127.0.0.1:$PORT/debian/dists/nosuite/asgard/binary-ppc64/Packages
HTTP/1.1 200 OK content-type: application/octet-stream Etag: "1750f481b8f1ae00-363" Last-Modified: Wed, 29 Mar 2023 17:38:19 GMT Content-Length: 867 Accept-Ranges: bytes Date: Fri, 14 Apr 2023 16:03:49 GMT Server: Python/3.10 aiohttp/3.8.4
HTTP/1.1 200 OK content-type: application/octet-stream Etag: "1750f481b8f1ae00-363" Last-Modified: Wed, 29 Mar 2023 17:38:19 GMT Content-Length: 867 Accept-Ranges: bytes Date: Fri, 14 Apr 2023 16:03:49 GMT Server: Python/3.10 aiohttp/3.8.4
HTTP/1.1 200 OK content-type: application/octet-stream Content-Encoding: gzip Vary: Accept-Encoding Etag: "1750f481b8f1ae00-363" Last-Modified: Wed, 29 Mar 2023 17:38:19 GMT Content-Length: 867 Accept-Ranges: bytes Date: Fri, 14 Apr 2023 16:03:49 GMT Server: Python/3.10 aiohttp/3.8.4
HTTP/1.1 200 OK content-type: application/octet-stream Etag: "1750f481b8f1ae00-6d9" Last-Modified: Wed, 29 Mar 2023 17:38:19 GMT Content-Length: 1753 Accept-Ranges: bytes Date: Fri, 14 Apr 2023 16:03:49 GMT Server: Python/3.10 aiohttp/3.8.4
Note that uncompressed data (Packages) will be compressed during transmission when Accept-Encoding is 'gzip'. This change should not affect tests based off pulp-smash, but if they do break it is more likely the test was working around this issue.