“HTTP Error 431 Request Header Fields Too Large” on large file uploads

[mariusschroeterCP]

Describe the bug The larger a file, the more upload resources are requested by the server for the upload via POST with the TUS protocol. As a result, the size of the Upload-Concat header grows proportionally to the file size in the final request and can therefore exceed the maximum permitted header limit of the server or proxy, which triggers the “HTTP 431 Request Header Fields Too Large” error.

You can increase the chunk size but this only postpones the problem.

To Reproduce Define the Uploader with a small chunk size (like 8mb) and upload a large file (probably bigger than 8gb). Check the last Post Request Upload-Concat Header.

Expected behavior The implementation of the TUS protocol in GitHub - tus/tus-js-client: A pure JavaScript client for the tus resumable upload protocol takes a different approach. Here, only as many upload resources are requested as parallel, simultaneous uploads are to be carried out. PATCH requests are used to repeatedly append chunks to the same uploadUrl, which is made possible by setting the upload offset. With Uploady, however, Upload-Offset is always 0, and the server requests a new upload resource for the next chunk. As a result, the size of the Upload-Concat header in the final request no longer depends on the file size, which in our view is recommendable and conforms to the TUS specification. It should also have a positive effect on the number of possible preflight requests.

Versions "@rpldy/tus-uploady": "^1.8.1"

[yoavniran]

thanks for the detailed issue description @mariusschroeterCP . I will try to investigate and provide a fix soon

[github-actions[bot]]

It's been a while. Waiting for an update... 🧐