[wpe-2.38] Overlapping samples from different contents are causing macro blocking

[varumugam123]

This issue is created to pursue the problem described in https://github.com/WebPlatformForEmbedded/WPEWebKit/pull/1184

Quick Summary :

• App plays 6 seconds Logo
• Then loads actual video. Before appending data, app offsets timestamps with (last logo sample PTS + last logo sample duration). This is because actual asset starts with PTS ~0.
• The asset's PTS starts with 0.083. Edit list implementation moves the samples PTS upward (i.e PTS : 0.083 => 0.000, DTS : 0.000 => -0.083)
• This causes logo samples & assets samples to overlap
• The existing logic webkit has to detect the overlap & remove the samples is suppressed by the existence of a sync sample
• This causes incorrect sync sample to be used by further samples for decoding causing macro blocks

The macro block is observed with wpe-2.38 (fc1703ed69) as well

Sample app : Uploading out-of-order-test-02.tgz.txt…

[cadubentzen]

@varumugam123 I tried your sample app but I couldn't reproduce it on the RPi3. I also had to decrypt big-buck-bunny-trailer-video-cenc.mp4 since it's CENC-protected but the EME bits were missing. Used the key from here.

• buildroot branch: R4_4. EDIT: also with the main branch
• WPE branch: wpe-2.38 (99973c66c)

Here's a screen recording from the Pi: https://github.com/WebPlatformForEmbedded/WPEWebKit/assets/11878456/593ebdee-9053-4cb2-ac0d-e4813285dde0

Could you share what you see on your side?

The only artifacts I can see here are those below, but they seem to come from the big-buck-bunny bitstream and also visible playing that asset alone with other players and browsers.

Maybe something is missing to reproduce what you saw originally.

[cadubentzen]

@varumugam123 Here's the unencrypted big-buck-bunny file that I used in my test, could you see if you reproduce the issue with it?

https://github.com/WebPlatformForEmbedded/WPEWebKit/assets/11878456/07cd2f42-97bb-4386-8bfc-fe01c77269f9

Additionally, could you also send me the GStreamer logs?

killall WPEFramework
export GST_DEBUG="3,webkit*:TRACE"
export GST_DEBUG_NO_COLOR=1 
export GST_DEBUG_FILE=/root/gst.log
WPEFramework -b &
# open example via the WPEFramework UI
# the logs will written to /root/gst.log

[varumugam123]

Let me capture a video of the issue with the test app and share here.

[varumugam123]

gst.log

https://github.com/WebPlatformForEmbedded/WPEWebKit/assets/29426674/8ef8da6f-bb47-4650-b984-c0c6470a9e03

Hi @cadubentzen Please find the attached video of issue reproduction & the gst logs as requested. Please note, the issue is reproduced on Broadcom based STB (Xumo Stream Box)

[cadubentzen]

Hi @varumugam123! Thanks for the log file and video. We couldn't yet reproduce this even on the sagem box that is also Broadcom and uses Nexus.

However, I was able to write a layout test in https://github.com/cadubentzen/WebKit/commit/f1fcd919f141ed7af52678638bbb1f31e6ae0a22 that reproduces the issue described in your analysis in https://github.com/WebPlatformForEmbedded/WPEWebKit/pull/1184, also upstream. Thanks for the in-depth analysis btw!

While the simple change in your PR fixes the new test case, it causes other failures in the MSE layout tests, so I need to dig deeper to see how to fix it and get it reviewed.

[cadubentzen]

PR open upstream and waiting for review: https://github.com/WebKit/WebKit/pull/27033 Once it lands, I'll backport it to wpe-2.38 here.

fyi @varumugam123

[cadubentzen]

Hi @varumugam123, the fix has been backported into wpe-2.38. Let me know if you're able to test it and whether we can close this issue. From my side, I was able to reproduce the issue locally on the desktop and RPi3 and checked that the fix works.