tuf.api.exceptions.DownloadLengthMismatchError when serving files with Flask

[Eoic]

Describe the bug I serve targets and metadata files with Flask, using gunicorn server. When I open the application, it tries to download and apply the update, but fails with DownloadLengthMismatchError. Correct file from the server is requested and size of the requested file in targets.json is correct. Anyway, when I serve files with http.server, it works as expected.

Here is a relevant section from targets.json:

{
  "APP-1.3.3.patch": {
    "hashes": {
      "sha256": "1e562f905761e3fa1288dd40b0ef362a85d76c1b71e417f319fd34939238230b"
    },
    "length": 36161823
  },
  "APP-1.3.3.tar.gz": {
    "hashes": {
      "sha256": "a3f9ee709ea92f4056e006c1c983f1bdac3a807933292d82c77adcee079eafae"
    },
    "length": **88540789**
  }
}

Here is a stack trace of the exception when trying to download an update:

Traceback (most recent call last):
  File "widget\windows\version_update.py", line 167, in handle_update
  File "widget\windows\version_update.py", line 173, in apply_update
  File "tufup\client.py", line 130, in download_and_apply_update
  File "tufup\client.py", line 226, in _download_updates
  File "tuf\ngclient\updater.py", line 254, in download_target
  File "contextlib.py", line 137, in __enter__
  File "tuf\ngclient\fetcher.py", line 105, in download_file
tuf.api.exceptions.DownloadLengthMismatchError: Downloaded 88651851 bytes exceeding the maximum allowed length of **88540789**

To Reproduce Steps to reproduce the behavior:

1. Serve repository metadata and targets files using Flask:

   @app.route('/targets/<path:path>', methods=['GET'])
   def targets(path):
   return send_from_directory(os.path.join(os.getcwd(), REPO_DIR, 'targets'), path)

2. Start gunicorn server: gunicorn -w 4 -k gevent --timeout 1800 --bind 0.0.0.0:8000 src.server:app.
3. Create tufup.client.Client instance.
4. Call check_for_updates(), and then download_and_apply_update().

Expected behavior

1. Update is downloaded successfully.
2. Target file size is the same as described in a relevant entry of targets.json metadata file.

Screenshots -

System info (please complete the following information):

• OS: Windows
• OS Version: 11
• Python version: 3.11.4

Additional context Here are response headers when fetching target file with curl. Response size seems to be correct:

< HTTP/1.1 200 OK
< Server: gunicorn
< Date: Mon, 18 Dec 2023 11:52:30 GMT
< Connection: close
< Content-Encoding: gzip
< Content-Disposition: inline; filename=APP-1.3.3.tar.gz
< Content-Type: application/x-tar
< Content-Length: 88540789
< Last-Modified: Mon, 18 Dec 2023 10:47:19 GMT
< Cache-Control: no-cache

[dennisvang]

@Eoic This is probably caused by the response header: Content-Encoding: gzip

This causes the file to be "unzipped" automatically (by requests) before python-tuf gets a chance to check the file hash.

The solution is to configure your server as follows:

• do not set any Content-Encoding header
• do set Content-Type: application/gzip