search

AirenSoft / OvenPlayer

OvenPlayer is JavaScript-based LLHLS and WebRTC Player for OvenMediaEngine.
https://OvenMediaEngine.com/ovenplayer
MIT License
505 stars 125 forks source link

Separate audio and video into different html tags #183

Closed GregOriol closed 7 months ago

GregOriol commented 3 years ago

Hi,

Thanks for your super awesome work!

I have a very special and disturbing request to submit today.

The situation is that I would like to have a streaming content on a webpage (like a person talking), as well as let users navigate other content and sometimes play it, could be audio or video. However, on Safari on iOS, it won't work: the browser allows only 1 video with audio playing at a time. So if the user clicks on play on a secondary content on their iPad, the streaming stops and only the secondary video plays. It is a known issue on Safari on iOS (https://bugs.webkit.org/show_bug.cgi?id=162366). It however works perfectly on any desktop browser...

A workaround seems to be to split the video into a <video> and an <audio> tags: then the video is muted and can play as a "background" video, while the audio is played separately, and multiple audios are allowed to play together anyway. This example provided in the comments of the webkit bug works like that: http://curious-electric.com/_temp/video-bug/workaround.html

It also seems to work like this with webrtc video-conferencing solutions (jitsi, ...).

So, would it be possible to implement something like an audio/video split with OvenPlayer and OvenMediaEngine to allow a streaming and another video together?

SangwonOh commented 3 years ago

Hi @GregOriol. Is it correct that I understand?

The issue Safari on iOS can not play multiple videos at the same time.

To solve issue... By separating the video audio tag of each video, we can make it behave as if it were playing multiple videos.

Thanks!

GregOriol commented 3 years ago

@SangwonOh Yes, instead of one <video> tag, using one <video> that has no sound or is muted and one <audio> with the sound allows "background" playing, which makes it possible to have another video content playing at the same time. Otherwise, Safari on iOS stops one "foreground" video to play another.

SangwonOh commented 3 years ago

@GregOriol Interesting. However, with the current OvenPlayer implementation, separating video and audio tags seems to be a very difficult task. This is because from the initial design, it was designed to play audio and video in a single video tag. For now, I think we have to think about whether there are other alternative methods.

GregOriol commented 3 years ago

In the meantime, if anyone else stumbles upon this situation, I have a little workaround that kind of does the thing on Safari iOS for a WebRTC stream.

On the player's stateChanged event, when catching a playing state, it is possible to separate the video from the audio by finding the video DOM element then getting the stream with video.srcObject, setting video.muted = true and pushing the stream into an audio for example context with createMediaStreamSource. Of course on other states and events, you'd also have to stop the audio, adapt the volume, ...

Then, any other <video> on the same page will play in parallel with the stream.

SangwonOh commented 3 years ago

@GregOriol Brilliant! Did you have any problems with audio video syncing when you did it that way?

GregOriol commented 3 years ago

That's why I was saying "kind of does the thing for a WebRTC stream": it seems to work well because the syncing is done by the stream and it's the same stream object being used. Not sure that would work for HLS/Dash for example.

SangwonOh commented 7 months ago

We are closing the issue due to a long period of inactivity.