Detect when audio is locked by Autoplay Policy

[oswaldofreitas]

I have some sounds that should be played as soon as the user hover some elements after loading the page. However there are some cases where the audio is locked by the Web Audio, Autoplay Policy and then I would like to show an icon to the user indicating that any audios aren't allowed to be played until he clicks somewhere.

Is there a way in Howler to get a flag like audioPolicy = 'suspended' or something else to indicate that it's waiting for a user interaction to be enabled?

[electromotif]

Howler exposes the audio context.

https://github.com/goldfire/howler.js#ctx-boolean-web-audio-only

The state property can be running, suspended or closed. https://developer.mozilla.org/en-US/docs/Web/API/BaseAudioContext/state

[oswaldofreitas]

That's "Web Audio Only", I need a safe way of knowing if Howler will be able to play a sound either on Web Audio or HTML5 audio

[electromotif]

Allright. Maybe try this?

"... you can use the play() method on the video and audio element to start playing your media. The play() method returns a promise in modern browsers (all according to the spec). If the promise rejects, it can indicate that autoplay is disabled in the current browser on your site."

https://stackoverflow.com/questions/49939436/how-to-detect-if-chrome-safari-firefox-prevented-autoplay-for-video

[oswaldofreitas]

yes, I've tried it and the result didn't match the state of Howler being able to reproduce the audio or not, trust me, I've tried almost everything. the play() doesn't reject the promise when it can't reproduce audio, there is a big thread in Firefox about that.

I really think it's a new feature to be added here.

[Mte90]

There are some workaround to detect if there is this issue? I saw that happens on firefox in the howlerjs demo as example, I have to change the permission to the domain to execute audio/video.

[Jimbly]

Yeah, this feature is definitely needed, especially since before being unlocked Howler seems to queue all of the sounds, so any rollover sounds, potentially hundreds will barrage the user as soon as they click on something. I added and exposed a safeToPlay() method in my fork, but it's amidst a bunch of other fixes/changes I needed, so not sure how useful it is generally ^_^.

[Mte90]

There is a pull request for this patch?

[Jimbly]

Not yet, it's amid a bunch of other fixes and refactors to fix what I needed for a project, so I'm stuck on my fork for now. Might not be too hard for someone to pull the bits needed, though I seem to remember I had to change how it tracked the suspend/resume state quite a bit.

[goldfire]

@Jimbly I haven't had a chance to look at your fork yet, but could you give a high level overview of your approach to detect this? One of the major issues with Google doing this when they did was that there wasn't a way to detect this. I submitted bugs for things like the promise not rejecting, which were marked as "wontfix." I've yet to see a workaround that can successfully detect this, so if there is one now I'd be happy to get it implemented into howler.

[Jimbly]

Howler mostly already knows when it's safe to play things, with WebAudio it's through the result of trying to play a 0s clip and seeing that it has ended, or the context reporting running at startup, I think. For HTML5 audio it's generating Audio elements in response to a user event. Mostly I just traced all of the different routes and exposed that state, and fixed a bunch of issues that came up as I was testing on various platforms.

[oswaldofreitas]

I've done this in my project:

export const isAudioLocked = () => {
  return new Promise(resolve => {
    const checkHTML5Audio = async () => {
      const audio = new Audio();
      try {
        audio.play();
        resolve(false);
      } catch (err) {
        resolve(true);
      }
    };
    try {
      const context = new (window.AudioContext || window.webkitAudioContext)();
      resolve(context.state === 'suspended');
    } catch (e) {
      checkHTML5Audio();
    }
  });
};

Sharing here since it could give you guys some clue on how to identify if sound is locked by audio policy.

[oswaldofreitas]

additionally, I have this:

const userGestureEvents = [
  'click',
  'contextmenu',
  'auxclick',
  'dblclick',
  'mousedown',
  'mouseup',
  'pointerup',
  'touchend',
  'keydown',
  'keyup',
];
const unlockAudio = () => {
  commit(types.mutations.SET_AUDIO_LOCKED, false);
  userGestureEvents.forEach(eventName => {
    document.removeEventListener(eventName, unlockAudio);
  });
};
if (await isAudioLocked()) {
  commit(types.mutations.SET_AUDIO_LOCKED, true);
  userGestureEvents.forEach(eventName => {
    document.addEventListener(eventName, unlockAudio);
  });
}

to set an UI indicator if sound is enabled or not

[giray123]

Does the code of @oswaldofreitas work? For me, it gives unstable results on Safari?

[F2]

Does the code of @oswaldofreitas work? For me, it gives unstable results on Safari?

To make the code work in Safari, you need to use Howler's context instead of a newly created context.

Additionally, it doesn't work if Howler's context is intentionally suspended, so you need to disable that.

Howler.autoSuspend = false;

export const isAudioLocked = () => {
  return new Promise(resolve => {
    const checkHTML5Audio = async () => {
      const audio = new Audio();
      try {
        audio.play();
        resolve(false);
      } catch (err) {
        resolve(true);
      }
    };
    try {
      const context = Howler.ctx;
      resolve(context.state === 'suspended');
    } catch (e) {
      checkHTML5Audio();
    }
  });
};

[kslstn]

I've tried all of the above and some other tricks trying to check the audio context. The onplayerror event is not fired when I attempt to play a sound without prior user interaction in iOS Safari.

What seems to reliably work for me now (iOS 15.7.3) is:

1. Set Howler.autoSuspend = false
2. Assume audio is locked until any interaction takes place. Show a UI element indicating that.
3. Do not attempt to play sounds until we assume the audio is unlocked. Otherwise, every attempt can get played at once after the audio is unlocked, as @Jimbly reported. In my case that's a multiplication of one sound file, which turns out to be very loud.
4. Add a user interaction event listener. The handler for it is a sound.play() with a new Howl({ src: 'my-path', volume: 0})
5. Assume the audio is unlocked, hide the UI element indicating the lock.
6. Play sounds 🎺

I understand Howler is supposed to do step 4 by default. Maybe it's because I added some stopPropagation()s to my event listeners, but this didn't seem to work at all in my project.

[ollien]

@kslstn

I understand Howler is supposed to do step 4 by default. Maybe it's because I added some stopPropagation()s to my event listeners, but this didn't seem to work at all in my project.

No, I don't think this is your doing. Near as I can tell, this is Howler's doing. playerror is only emitted for HTML5 audio, so if you're using Web Audio, the example you linked won't work.

https://github.com/goldfire/howler.js/blob/9610df82fb93467d62f25a7b1682d534923dc963/src/howler.core.js#L899-L969