Lag, Blocking & visual glitches on firefox

[thet0ast3r]

Hi,

We recently switched to paella player 7 with opencast, and identified a range of problems with it, most likely (at least somewhat) related to each other.

Issues that occur, most prominent on firefox:

• when quickly navigating with the playback bar, the playback bar lags behind in loading the video, cumulatively.

To reproduce: https://paellaplayer.upv.es/#/demos/hls-multiaudio-smil-1 play this video (using firefox), and quickly jump from the beginning to the end and back to the beginning with the mouse. After some quick repetitions, the player is behind and the playback bar starts jumping on its own.

• on some systems (also mainly ff), when one tries to drag the playback bar, it sometimes gets stuck, resuming playback from an unwanted position (very undesirable in long videos)
• sometimes the 'mouseup' seems to not be registered when using the playback bar. This results in the user unwantedly changing the playback position by moving the mouse after intended seeking.
• hovering over timeline previews quickly can cause a large number of aborted requests for preview images (often multiple of the same request url)
• preview images tend to get stuck when moving the mouse outside of the player container quickly/to the side. They then block the view of the video feed(s).

On my system, these issues only have a minor impact on usability (firefox 120, mac os 14.1.0, apple m1 pro), but we got reports from users that said it was nearly impossible to use the player (firefox on linux). I sadly was unable to pinpoint the exact cause of the problem, but there definitely seems to be some type of unintended behavior that limits the seekability of videos.

Additional info:

• Mostly videos with 2 streams are affected.
• Our videos are served via hls(wowza) and we use opencast-paella-player.
• single-stream videos without timeline previews seem to work mostly fine (also hls with wowza).
• users that had problems with our videos also had problems with the paella demo video abeit to a lesser extent (maybe/probably due to the short video and lesser amount of timeline previews)

Please let me know if you need any additional/specific information.

thanks :)

[karendolan]

@thet0ast3r in case it helps, our site uses these additional HLS.js params to help with buffering.

lowLatencyMode: true, backBufferLength: 90, startFragPrefetch: true,

Added to the defaultHlsConfig list. https://github.com/polimediaupv/paella-core/blob/main/src/js/videoFormats/es.upv.paella.hlsVideoFormat.js#L11C14-L57 (FYI - Some of the params on the current defaultHlsConfig list are not used in the latest HLS.js)

[karendolan]

@thet0ast3r also our site override the paella-core es.upv.paella.hlsVideoFormat.js file to deal with iPad and multiple HLS video events. iPad allows using the HLS.js library for single video events, but for multiple video events it uses it's native HLS player. I'm curious if the people having the blocking issue at your site are using iPad versus LapTop/DeskTop?

[thet0ast3r]

Hi, thank you for the reply. I will try different hls params. What makes me think it is not network related is the fact that we never had little issues with paella player 6, and e.g. on other browsers or on other OS's these issues are way less severe. Like i said, on my machine it very hard to trigger unintended behavior, but on other machines/configurations (say x64, ubuntu + firefox) the player is unusable. We have not gotten any reports from mobile users yet, so far only desktop browsers. Safari seems to not be affected very much, chrome seems to have a few more issues.

[karendolan]

@thet0ast3r something to try to check if the issue is aggressive multi-video event synching, when both videos have not had a chance to finish buffering.

1. Load your video with param "&logLevel=DEBUG" at the end of the URL
2. Open your browser inspector/debugger console log
3. Perform the behavior of dragging the playback bar until the unwanted time issue appears
4. Check the browser console logs for text "Video synchronization triggered" If a lot of "Video synchronization triggered" log lines appear close together after a failed seek, there may be an issue with the waiting for the video to be ready after buffering.

TMI: This suggestion is related to an issue our site had where we had to adjusting the readystate wait level by separating out the FireFox check to it's own conditional. The multi-video synch code: https://github.com/polimediaupv/paella-core/blob/main/src/js/core/StreamProvider.js#L172-L183 The default ready state is set low, at 2. But not all browsers can have their currentTime set when they are at level 2 when their readystate falls during a seek buffering: https://github.com/polimediaupv/paella-core/blob/main/src/js/videoFormats/es.upv.paella.hlsVideoFormat.js#L310-L321

[thet0ast3r]

ok, i did some testing with loglevel debug:

it seems to be related to the way the player syncs. Only after the last 'video synchronization triggered' appears, the playback bar returns to normal.

This is what it looks like, is it unusual?

so if i change the readystate to something higher than 2, i might alleviate these ui issues? To me it looks like the playback bar is coupled to the actual current position in the video, and it also appears to be that all events (moving/Dragging the bar) are being handled sequentially. Is there a way to just throw away (kind of debounce) all those unnecessary seeks?

Also: i thought i disabled the cookiedataplugin, but apparently it still writes the cookies very very often, i'll have to take a look at this, too.

[karendolan]

so if i change the readystate to something higher than 2, i might alleviate these ui issues?

Yes, for our site, setting the higher readyState worked. We kept the exception for FireFox.

1. The sync code calls setCurrentTime

       this.player.log.debug("Video synchronization triggered");
       secPlayer.setCurrentTime(currentTime);

2. The setCurrentTime(currentTime) does a waitForLoaded()

       if (this._videoEnabled) {
           await this.waitForLoaded();
           return this.video.currentTime = t;
       }

3. The waitForLoaded() does the check (this.video.readyState >= 2), as mentioned in my earlier post.

Conclusion: Giving the video extra time to buffer by reaching a higher readyState on every seek, preserves the seek sequence, and prevented undesired seek behavior for our site.

Ref for setCurrentTime's calls to waitForLoad() https://github.com/polimediaupv/paella-core/blob/main/src/js/videoFormats/es.upv.paella.mp4VideoFormat.js#L97-L106

[ferserc1]

The example configuration of the HLS plugin was extracted from a fairly old version of the hls.js library, which apparently in recent versions caused problems.

Although it is not a paella-core issue in itself, I have modified some of these example configuration settings so as not to be misleading. However, I'm going to use the information in this thread to update the documentation about the HLS plugin

[ferserc1]

I have added a new parameter in the configuration to control the maximum desynchronization time of the videos. It is available starting from version 1.46.2

[karendolan]

@ferserc1 is the new config option for widening the re-synchronization time of the videos a way to give FireFox more time to buffer a lagging video? Are you concerned that monitoring the ready-state of the lagging video is not good enough to determine if the second video is ready for a new seek?