search

Dash-Industry-Forum / dash.js

A reference client implementation for the playback of MPEG DASH via Javascript and compliant browsers.
http://reference.dashif.org/dash.js/nightly/samples/dash-if-reference-player/index.html
Other
5.08k stars 1.67k forks source link

Utilize webworkers if available #3474

Open wilaw opened 3 years ago

wilaw commented 3 years ago

Allowing media playback to occur outside of the main thread can have many performance benefits, especially when the the main thread is under heavy contention. MSEv2 is bringing worker support to the standard and it is already implemented in Chromium 88.0.4300.0 or later with chrome://flags/#enable-experimental-web-platform-features enabled.

MSE in workers explainer: https://tinyurl.com/mse-in-workers MSE in workers demo: https://tinyurl.com/mse-in-workers-demo

Request is for dash.js to detect the availability of webworkers and if available, to transfer the MSE operations to a dedicated worker for optimum performance.

image

lamflam commented 3 years ago

This is actually somewhat critical for my use case where lowest possible latency is the goal. The issue isn't necessarily the performance of dash.js itself, but that playback is subject to the performance of all other javascript running on the same page.

I'm able to consistently get a smooth stream with ~700ms latency (audio only) however as soon as an update comes in and I need to re-render a portion of the screen it stalls buffering and the player pushes back from the live edge significantly. There may be some settings to make this less significant, but I know when previously using hls.js the fetch/buffer load work always ran smooth because it was done in a worker.

I'm considering making an attempt at this, but I'm not yet familiar with this code base, I've only browsed through so I would appreciate if anyone has any tips or ideas on how to approach or where to get started.

dsilhavy commented 3 years ago

@lamflam That is interesting. We did some optimizations in dash.js 4.0.0 to reduce the load and the memory consumption. Just to be precise here:

The idea described by @wilaw is to move the mediasource handling in webworkers. This is a new feature of Chrome and still being improved and worked on (as far as I know). I would be surprised if hls.js is already using this. However, I think hls.js is doing the transmuxing (conversion from ts to mp4) in a separate worker. This step is not required by dash.js.

It would be interesting to understand what causes the performance issues in your case. It might also be the size of the manifest since parsing from XML to JSON can take some time.

In case you want to look into MSE in webworkers I suggest to checkout the following classes:

I have not checked the MSE webworker implementation yet so I can not really comment on the overall effort. In general, this is something we definitely want to add to one of the upcoming releases. We will have a sync call with DASH-IF mid July to define the scope of the next sprints.

lamflam commented 3 years ago

@dsilhavy I plan to upgrade to v4 later this week and can report back if it makes any noticeable difference. However I don't expect it to mainly because at the root it's not a performance problem with dash.js, just the fact that dash.js and application rendering are happening on the same thread in combination with our latency/buffering requirements. We are using segment templates so the manifest XML is quite small and is only downloaded once at the beginning, but such low latencies mean that even a small blip in sending the current mp4 to the buffer is likely to cause some stuttering. If both fetching and writing to the buffer happened in the worker I think it would entirely alleviate the issue (of course I haven't been able to test and confirm that though).

I haven't dug into hls.js, but what you said does make sense that it may be transmuxing only that they do on the worker. It's more likely that the reason I wasn't seeing stuttering issues with hls.js is that I was forced to download and buffer larger segments which allowed for more contention with rendering without emptying the audio buffer. This is the same reason the latency was much higher though and why we moved to dash.js with fragmented mp4.

Thank you, I will start taking a look at the classes above, and if you have any further thoughts on the specific approach they'd be appreciated. If I actually get anywhere with this I will open a PR/issue to get some feedback.

chrisn commented 2 years ago

MSE in Workers is a feature that we're interested in using. I note that there's now an Origin Trial open in Chrome 95 to 99.