search

videojs / video.js

Video.js - open source HTML5 video player
https://videojs.com
Other
37.94k stars 7.43k forks source link

Subtitle Download Behavior #5252

Open jaruba opened 6 years ago

jaruba commented 6 years ago

Description

When adding subtitles with addRemoteTextTrack({}, false) it always downloads the subtitles immediately. I understand that this is because videojs wants to know if the subtitle type is supported and can be correctly parsed, but as I'm adding a lot of subtitles at once, this effectively freezes the page for some time (especially on phones) and puts an unnecessary strain on the server.

I would rather prefer the subtitles being downloaded when the user actually selects one, even if it leads to an error. Is there a plugin or a known way to accomplish this?

Steps to reproduce

Explain in detail the exact steps necessary to reproduce the issue.

  1. Let a video start playback
  2. Add a lot of subtitles with addRemoteTextTrack({}, false)
  3. Check Network tab in the console

Results

Expected

I would expect it to not download a gazillion subtitles at once. None of them actually, as their not even needed yet and may never be needed, the user most likely needs 1 specific subtitle or even none.

Actual

It downloads a gazillion subtitles at once.

Error output

None

Additional Information

Please include any additional information necessary here. Including the following:

versions

videojs

v7.0.3

browsers

All

OSes

All

plugins

None

OwenEdwards commented 6 years ago

This is really an issue with the WebVTT processing component (vtt.js), and was flagged by @gkatsev in https://github.com/mozilla/vtt.js/issues/340. The downside of not loading all of the text tracks is that there would be a delay if the user chooses to switch captions or subtitles track during playback.

We're definitely looking at improvements to this, but it'll depend on changes in videojs-vtt.js, our fork of Mozilla's vtt.js.

gkatsev commented 6 years ago

First reported in #1913.

Yeah, is waiting for captions to load upon selection OK? A hard question to answer.

OwenEdwards commented 6 years ago

Ideally captions would be loaded and processed in the background, in a way that wouldn't block playback but would make sure that each caption cue is loaded and processed in time for them to be displayed. The best way to implement that is to mux them into an HLS, I would think, but I'm not entirely sure how video.js' VHS handles captions and subtitles in the HLS mux.

jaruba commented 6 years ago

I rapidly went through the history of this, i'll read it all more carefully later on as this is not the proper time for it. I prefer waiting subtitles to load on demand, yes, as compared to downloading 15-30 subtitles when you might not want a subtitle at all.

How is vtt.js blocking logic? Can't we just push that entire script in a web worker?

jaruba commented 6 years ago

All browser blockage aside, the biggest issue here is that I'm using a 3rd party subtitle host for this, and every play will cripple their servers more for practically no reason.

gkatsev commented 6 years ago

The parsing of the caption file is synchronous and if the caption file is large it will cause issues, also, if we're loading a lot it's also an issue. It's theoretically possible to offload it into a web worker but we haven't had a chance to investigate that yet. https://github.com/video-dev/vtt.js/issues/4

gkatsev commented 6 years ago

@OwenEdwards I think VHS only loads the selected track.

OwenEdwards commented 6 years ago

@jaruba I was just playing with moving vtt.js over to a Web Worker, but it's not partitioned from the DOM in a way that makes that easy. If you know more about this area, you're very welcome to help make the changes that would improve this situation.

jaruba commented 6 years ago

@OwenEdwards

but it's not partitioned from the DOM in a way that makes that easy.

Could you elaborate on this? Why does the web worker need DOM access at all? You just push blocking synchronous logic to it and it returns the result without blocking anything as it's a different thread.

You can even push simplistic blocking logic in a separate web worker, like searching for the correct text to show based on playback time if needed, all DOM related logic needs to be kept in the main thread obviously.

I'd be happy to take a look at it as long as you can explain what issues you've had with it, on which lines, to simplify my journey.

This is still just half of the issue for me though, https://github.com/videojs/video.js/pull/2192 would need to be re-implemented, it's way out of date atm. @gkatsev do you think there's anything salvageable from that PR? Or is it easy enough to re-implement for you in the scenario in which vtt.js stops being a blocking issue?

gkatsev commented 6 years ago

It's probably fairly easy to update that PR. Maybe update it and make it an option for the player?

I think vtt.js created the DOM element for the track as part of parsing.

OwenEdwards commented 6 years ago

Right - I thought the vtt.js parsing would be separate from the part which creates elements in the DOM, so I was looking to push that parsing chunk to a Web Worker. That's all I know, @jaruba, you're welcome to dig into it yourself - you seem to know more about it than I do!

jaruba commented 6 years ago

Thanks for all the help / hints.

This is a big enough issue for me to dedicate time for it. It's silly that something as simple as subtitle parsing can go so horribly wrong and not be fixed for so long. If there's any way to fix this damn thing, I'll find it, and go all voodoo on it's ass.

5782662a-ec62-11e5-8c4c-902f7b72c3bd

61ba1b74-ec62-11e5-9c19-9398955c7f48

6ef10bae-ec62-11e5-951b-d13cbd0bb0c0

stale[bot] commented 6 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Chocobozzz commented 6 years ago

I think this issue should not be closed. Downloading and parsing all vtt files is a waste of bandwidth, and could be a real issue for popular videos that have a lot of subtitles. An option when adding subtitles to video.js like lazy for example would be ideal (like @gkatsev said).

gkatsev commented 6 years ago

Yeah, this issue should be pinned.

bvibber commented 5 years ago

We're reviving our frontend move to video.js for Wikipedia, and currently one of the last blockers is this; it's a performance issue as we may have a couple dozen subtitle languages for popular videos while usually none or one will be used, and the fetches go through an API endpoint which while cacheable would have middling worst-case performance.

We'd prefer to take a download & parsing hit on language switch in favor of the optimistic downloads on setup; is this something we should target to be the default in a patch, or make the behavior switchable?

JohannesKuehnel commented 3 years ago

@gkatsev Any news on the asynchronous parsing in 2021? Seems like the parsing of large .webvtt chunks (~1-2MB) is still taxing on the main thread.

gkatsev commented 3 years ago

Unfortunately, we haven't had the bandwidth to make this run in a web worker or to make the algorithm run asynchronously.

jaruba commented 3 years ago

I can't believe this is still a problem after more than 3 years since I made this issue.. Took me about 3-4h (and quite a few headaches) in a drunken daze to rewrite the subtitle handler back in 2018 to make it async.. If the devs at Firefox would of replied to my PR it would of been fixed a very long time ago. Is Brightcove hiring? 😄

ayi-afk commented 2 years ago

Still now fixed >, < so you say it's just enough to hack into vtt.js -> async ?