goldfire / howler.js

Javascript audio library for the modern web.
https://howlerjs.com
MIT License
23.98k stars 2.23k forks source link

HowlerJS does not work with video on iOS #247

Closed galinawind closed 8 years ago

galinawind commented 9 years ago

on iOS if you have a web page with an audio file and then you dare to play a video, the audio stops working after the video completes. I have a completely self-contained example with media that I can e-mail to anyone interested in helping out with this. All you have to do is hit "play sound", the sound will play. Then hit "add video", the sound will still play. Then hit "play" on the video to play it, no autoplay, this is iOS. Then try hitting "play sound" again. You will get nothing on iPad2 and distorted, garbled sound on iPad Air. I posted the code here, but it will not work without the media files. There is nothing to the code, it's 84 lousy lines. https://gist.github.com/anonymous/c564528ef81787245dea

If you reload the page, then everything kind of works. This is only an issue on "first entry" to the page. However having the user to navigate to a page and then reload it before they do anything else is not acceptable.

galinawind commented 9 years ago

Howler appears to have an issue with playback even when it's used together with a simple HTML5 video tag on iOS. Here is a very simple example. You can download the media for it here or you can use your own. https://www.dropbox.com/s/r5a8qmg0xnbnsi9/itScreensStyler_new.zip?dl=0 The source code is included here

<html>
<body>

<script src="howler.js"></script>

<video width="560" height="420" controls>
  <source src="small.mp4" type="video/mp4">
  <source src="small.webm" type="video/webm">
Your browser does not support the video tag.
</video> 

<button onclick="play_sound()">play sound</button>
<button onclick="stop_sound()">stop sound</button>

<script>

var sound = new Howl({ urls: ["Rayman_2_music_sample.ogg", "2240.mp3"] });
function play_sound()
{
    sound.play();
}
function stop_sound()
{
    sound.stop();
}

</script>
</body>
</html>
goldfire commented 9 years ago

I haven't had a chance to test this yet, but have you tried to force html5 audio to see if it is an issue with Web Audio specifically?

galinawind commented 9 years ago

I have tested with straight-up HTML5 audio and it does not appear to have an issue with this. This has to do with a known problem in iOS (since iOS 6 and still not fixed) where the sound gets garbled if you play video with a different sample rate than your audio. So if you play a video that has 48kHz audio in it and then you go play audio that has 44.1kHz the audio will be garbled or completely silent. The video can even be in a different tab and different site. There is a global sampling rate variable in Safari that, alas is read-only and is not initialized by default (the iOS bug). So I don't know what you can do about it. We have a workaround of checking the sample rate and then forcing a reload if it doesn't match the source sampling rate. This is not an issue on PC or Mac with all sorts of browsers. I'm not sure if it's an issue on Android, but I don't think it is. I don't know how to force Howler to use HTML5 audio. We have a rule in the existing code base that there may not be any browser-specific code in it, unless it is encapsulated in an external library like Howler or VideoJS or Greensock, or whatever. You may want to default to HTML5 on iOS if you are able to ....

Date: Sun, 7 Dec 2014 22:46:57 -0800 From: notifications@github.com To: howler.js@noreply.github.com CC: galinawind@hotmail.com Subject: Re: [howler.js] HowlerJS does not work with video on iOS (#247)

I haven't had a chance to test this yet, but have you tried to force html5 audio to see if it is an issue with Web Audio specifically?

— Reply to this email directly or view it on GitHub.

                  =
galinawind commented 9 years ago

After extensive R&D into this problem these past few days, we think it could potentially be solvable with minimal disruption to our web applications if only the Howler library provided a (hopefully performant) way to refresh the entire webAudio chain from the topmost audio context, ctx. We can create a new webkitAudioContext and check its sampleRate against the sampleRate Howler obtained when Howler was first src-included, and thereupon instantiated, in our page. This is how we can easily detect the situation when webAudio disappears subsequently: the two sampleRate 's, that of the original Howler.ctx and that of the new webkitAudioContext object 's (created with New after playing video on iOS), do not match. But now what if, instead of merely using a newly created audio context as such to detect this scenario there were instead a way to fix it simply by connecting-up the new web audio context-with-the-updated-sampleRate as the destination for all of our existing Howl objects' nodes? If it seems that this might work, would this be possible for someone to do who know much more about webAudio and the innards of Howler.js to try? It would probably be just one more utility function in Howler.js--I don't know if penalties for decoding and so forth need be incurred again or not for this rewiring.

As it stands, our hack right now is to detect the scenario of broken audio as above and then reload the page; but this is really bad because we stand to lose much state potentially (our large application is all on a single page) and create a really disjointed user experience just in the name of preserving our meticulously crafted audio design built around webAudio via Howler for mobile.

It would be much better if the problem could be handled more gracefully all behind-the-scenes by a simple function call in Howler that does not require a page reload. We could still detect the scenario like we do now--we do so upon window.onfocus and videojs on.play events (the latter because any video with 48 kHz audio embedded, even if paired with a matching 48 kHz Howler audio source, was tested to break on iOS ostensibly because the test devices we used are 44.1 kHz default audio playback devices and any media sample rates other than 44.1 kHz trigger the issue). We would just benefit greatly by an alternative to page reload. Merely rewiring webAudio back to a working state without the user having to know about it would be ideal if at all possible.

We are using Howler, after all, primarily for the feature of simultaneous audio source playback capability on mobile, particularly on iOS, afforded by webAudio; it needs to work well on these systems for our applications.

Date: Sun, 7 Dec 2014 22:46:57 -0800 From: notifications@github.com To: howler.js@noreply.github.com CC: galinawind@hotmail.com Subject: Re: [howler.js] HowlerJS does not work with video on iOS (#247)

I haven't had a chance to test this yet, but have you tried to force html5 audio to see if it is an issue with Web Audio specifically?

— Reply to this email directly or view it on GitHub.

                  =
DFortun81 commented 8 years ago

Any progress on this? The mobile QA team for our app has been pushing "garbled audio" bugs to our redmine and there really isn't anything we can do about it without directly modifying the HowlerJS library to implement the fix galinawind posted.

goldfire commented 8 years ago

If you want to create a new AudioContext, this is actually now built into 2.0 (sorry, forgot to reply to this). If you call `Howler.unload()``, it'll actually destroy the current AudioContext and create a new one.

heand commented 5 years ago

Hi, I have the same issue. If I play a video from the html5 video tag. And then start an Audiofile, the audio gets destorted on ipad (not desktop PC). I can set html5:true, but then i cannot control the file volume etc on the audio. Audio and Video are supposed to play together, this is a tool for listening training.. Any ideas?