Full asset is downloaded in progressive playback with "chunked" transfer encoding

[filipe-norte-red]

When a server transfers a video resource using "chunked" transfer encoding, wpe will download the full file, irrespective of size (e.g. 150MB). However, if "chunked" transfer encoding is not used, wpe will download small parts, progressively requesting further parts of the file as needed. The attached test.zip contains:

• test.html - Sample page that plays a video file
• myserver.js - Node based server that serves the test page and video asset. It uses https://github.com/pillarjs/send

Repo steps:

• Clone https://github.com/pillarjs/send and do the installation steps described on the projects page
• Copy myserver.js to the root folder of the local "send" project
• Copy test.html to folder "public" inside the local "send" project
• Download http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4 and save as "BigBuckBunny.mp4" inside the "public" folder (you can use any other asset with ~20MB size or more, but need to update test.html accordingly)
• Run "node myserver.js"
• Open the "http://localhost:8000/test.html" using wpe (confirm transfer pattern using web inspector)

If you'd like to disable chunked transfer mode for test, edit myserver.js and comment line 20: res.setHeader('Transfer-Encoding', 'chunked');

[magomez]

This happens on wpe-2.38 so we're using libsoup3 and HTTP2 requests. According to google, Transfer-Encoding:chunked is deprecated when using HTTP2 as all of the requests are chunked by design. The server shouldn't be replying with a Transfer-Content header when using an HTTP2 connection. Then, we may have a bug on our side handling that incorrect header that's causing that the full file is being downloaded. I'll ask internally about it.

[magomez]

Mi previous comment is not valid because the request is HTTP 1.1 and not 2.0. The problem seems to be that when Transfer-Encoding chunked is used, the Content-Lenght of the header is not available. Due to that, WebKitWebSourceGStreamer assumes that the content is not seekable and downloads the whole file. A way to handle chunked data should be added.

[eocanha]

I coded a patch to honor Content-Length when the Transfer-Encoding is chunked (so the WebKitWebSrc element has an idea of the actual length, considers that the content isn't "live" and can therefore unblock the seek and partial download features) and submitted it upstream for review as https://bugs.webkit.org/show_bug.cgi?id=266691 / https://github.com/WebKit/WebKit/pull/22093.

[filipe-norte-red]

Thanks for the feedback, @magomez . Will that be done via an upstream issue to be created? Or is it something you are working on?

[modeveci]

@filipe-norte-red , The PR is in review, in the meantime you can also give a try: https://github.com/WebKit/WebKit/pull/22093/commits/68c0a93e3bd650f84fe2a0a399b32d020d101ff2 Regards Ozgur

[eocanha]

The fix landed upstream as https://github.com/WebKit/WebKit/commit/dc4698316b36833706bc93aff6ad7a18cb3e4c33 and was backported to wpe-2.38 as https://github.com/WebPlatformForEmbedded/WPEWebKit/commit/b56a9c7fd3d95058fdacf5c004b6f21c0b33526c and to wpe-2.42 as https://github.com/WebPlatformForEmbedded/WPEWebKit/commit/4e6f7033a2cc06c36705d21b708cd4670630edb1. Closing issue.