Support for showing multiple video streams

[LukasKalbertodt]

Hi there!

Have you considered supporting multiple video streams? For example, this is quite common in the educational context with lecture recordings: usually, there is a recording of the lecturer and a recorder of the screen. Of course, one could combine both video streams somehow, but that's not ideal. Keeping both streams separate gives the user the choice of how to arrange the videos and how large each one should be displayed. This is mostly about two video streams -- there are cases with more than two streams (e.g. chemistry lecture with several experiment go pros), but those are rare exceptions.

I'm working in the field of educational videos and as far as I am concerned, we are somewhat lacking a good player for two video streams. There are players, like the Paella player, but quite frankly, I think the existing solutions (that I know of) are not really great :/ So having a really good player with support for two video streams would be great.

I am quite aware that supporting more than one video stream is quite involved and not a feature one can easily add as an after-thought. So I understand if you #wontfix this due it being out of scope. I still wanted to at least bring this up :)

[mihar-22]

Thanks for sharing @LukasKalbertodt. I had a quick look and it seems what you're after is keeping multiple video player instances partially in-sync. For example, in the case of a presentation you want playback and time position in-sync, but you could possibly want the screen recording muted (in the case there's audio for some reason) and the lecturer with volume. In general, you want media state between potentially multiple player instances in-sync. Feel free to correct me if I'm wrong 😄

Our <vds-media-sync element is built to do exactly that. It's still experimental and the API needs to be fleshed out, but the idea is as follows:

<vds-media-sync shared-playback shared-time shared-volume>
  <vds-video-player src="..." />
</vds-media-sync>

<vds-media-sync shared-playback shared-time shared-volume>
  <vds-video-player src="..." />
</vds-media-sync>

<!-- Does not share volume state. -->
<vds-media-sync shared-playback shared-time>
  <!-- Mute screen recording. -->
  <vds-video-player muted src="..." />
</vds-media-sync>

The currently implemented shared-playback behavior does not align with the name. A better name for it would be single-playback, hence why it's marked as experimental. I think this API can get us towards what you're looking for. Does it sound about right?

Also, in the Paella example the scrubber displays a preview video with the screen recording as you hover over it. This is also possible with the vds-slider-video element, which would simply point the src at the screen recording video.

This is just my very quick initial brain dump looking at it, I could be totally off.

[mihar-22]

Fullscreen is where things get tricky, but we could discuss alternative solutions when we get there.

[LukasKalbertodt]

Oh hey, thanks for the super fast reply!

keeping multiple video player instances partially in-sync. For example, in the case of a presentation you want playback and time position in-sync, but you could possibly want the screen recording muted (in the case there's audio for some reason) and the lecturer with volume. In general, you want media state between potentially multiple player instances in-sync.

That's correct. But further, there should only be one set of controls (one scrubber, one play button, ...). And it would be great if there were some controls to let the user change between different layouts (sizes/positoins of the single streams). And yep, fullscreen is probably a bit tricky.

I will play around with the code snippet you provided to see how well that works. To be honest, so far I haven't read very much about this player -- the "Documentation is currently not available." in the README stopped me :D But I will try to look into it and play around with the project in the coming days. I will report everything here.

[mihar-22]

That's correct. But further, there should only be one set of controls (one scrubber, one play button, ...). And it would be great if there were some controls to let the user change between different layouts (sizes/positoins of the single streams). And yep, fullscreen is probably a bit tricky.

Yep with our player you can still present one set of controls and sync it up. I'll try to setup an example for you once I get through the Beta release stuff. The laying out part might be application-specific, but we can talk about a generalized solution that we might be able to include in the library.

I will play around with the code snippet you provided to see how well that works. To be honest, so far I haven't read very much about this player -- the "Documentation is currently not available." in the README stopped me :D But I will try to look into it and play around with the project in the coming days. I will report everything here.

Quick warning that the API I showed is not the current state of the element, but I can easily get it there. I'll create a discussion space for it soon, and we can test drive it there. Your feedback as we build the sync element would be awesome.

[LukasKalbertodt]

in the coming days.

— me, March 9th

Hey, I finally got to play around with this library a tiny bit. Unfortunately, MediaSync (I'm using the React API) does not seem to work. At least not fully. For example, test this page: https://www.vidstack.io/docs/player/react/components/media/media-sync/#single-playback The single playback example doesn't work for me, neither in Chrome nor Firefox. Is that a known bug? The shared volume works though.

And judging from the docs, it seems like currently the kind of synchronization I want is not supported? You said it's not yet implemented but "I can easily get it there". Is that still the case?

[mihar-22]

Hey @LukasKalbertodt!

Hm, that's super strange. Not sure why it would randomly stop working - I'll need to look into it. I'm just wrapping up some site updates and then I'll have a peek either today/tomorrow.

And judging from the docs, it seems like currently the kind of synchronization I want is not supported? You said it's not yet implemented but "I can easily get it there". Is that still the case?

I should be careful with my words. I don't think the implementation is easy itself but the architecture is there to easily support something like it (if that makes sense). My plate is full right now and I don't think go down that tunnel right this second, but it's definitely on the roadmap.

You can have a look at media-group by the Mux team. This library attaches to the native media element so I think it should work together with Vidstack Player just fine. Let me know how you go.

[LukasKalbertodt]

Thanks for the quick answer and the information. I will take a look at media-group in the meantime. I fully understand you have lots of other stuff to do; thanks for your work on this!