detect `Unhandled Promise Rejection: NotAllowedError` when calling `.play()`

[lisongx]

https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/play

I found that the .play() by the media element interface method return a promise, which can reject due to some permission issue. Right now in safari 11 desktop, the auto play is disabled by default, so I can't this exception: Unhandled Promise Rejection: NotAllowedError. Any way I get catch this while using soundmanager2 ?

The recommanded way by apple using the media element is: ( https://webkit.org/blog/7734/auto-play-policy-changes-for-macos/ )

var promise = document.querySelector('video').play();

if (promise !== undefined) {
    promise.catch(error => {
        // Auto-play was prevented
        // Show a UI element to let the user manually start playback
    }).then(() => {
        // Auto-play started
    });
}

@scottschiller

thanks in advance!

[scottschiller]

Good observation, thanks! I'm interested in working something into SM2 for this case.

Yes, this is quite new for Safari 11 (on desktop) and has been out in Chrome since V.50. I wager others will follow suit soon. Mobile (iOS) Safari probably does similar, these days.

I currently have an onerror() callback for sounds. I may be able to extend that error to also accept a promise and start passing in an object with, say, { promise_rejection: true, promise_error: NotAllowedError } or similar.

That way, users could call something like

mySound.play({
  onplay: function() {
    console.log('Yay, playing');
  },
  onerror: function(errorCode, description) {
    // maybe failure happened _during_ playback, maybe it failed to start.
    // depends on what is passed to the function.
    // errorCode is currently based on W3 specs for HTML5 playback failures.
    // https://html.spec.whatwg.org/multipage/embedded-content.html#error-codes
  }
});

So perhaps I can extend errorCode to also include NotAllowedError and NotSupportedError types. It would be nicer if I normalize this stuff for users, so maybe I return an object with properties that are simpler.

So then it could be more like onerror: function(e) { if (e.notAllowed) { ... } { else if (e.notSupported) { ... } };

I'll look to get this into the next release.

For the record, I have been advising against people using auto-play for years. Mobile Safari and others have been blocking it for a long time, and now desktop is just catching up because it is equally annoying. ;) http://www.schillmania.com/projects/soundmanager2/doc/technotes/#mobile-device-limitations

[RyanK-CloudCover]

+1 on this issue, it's causing major issues for our Safari users even though they are not actually "auto-playing" when the page loads. They are hitting a play button. Need a fix to this ASAP, appreciate your immediate attention to this if possible. Thank you!

[scottschiller]

RyanK: Can you elaborate? SM2 does not expect a response from Audio().play(), so Safari's recent change to now return a promise (similar to Chrome's) should not break audio playback within the context of SM2.

You are the first to report this - so there could be a more widespread issue, or it may be something specific to the way you're initiating playback (or perhaps the parameters provided to SM2.)

Perhaps something has changed in Safari that is now blocking your click -> play actions, thinking they are auto-play or similar. From what I've seen, you must start playback via play() immediately from a user event like touch or click. No setTimeout(), etc., allowed.

If you have a live example where I can test and repro, that's a bonus. Console logs might help also. If you're seeing Unhandled Promise Rejection: NotAllowedError, then it's likely that Safari thinks you are trying to play audio from a user action as described.

[RyanK-CloudCover]

Thanks for the quick response. Yes the issue is we aren't doing a click and then immediately invoking play(). We have to do some other processes (things like registering/activating the device in our DB, checking permissions, checking their other active devices, etc) before it actually gets to the play call.

The console error is:

Unhandled Promise Rejection: NotAllowedError (DOM Exception 35): The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.

And the line it gives me is

play - soundmanager2-nodebug-jsmin.js:665:121

If I then manually skip to the next track, however, it will then play the next track, but letting it try to go to the next track automatically by itself will result in the same behavior where it doesn't play.

[kopf]

Yep, we're also having the same problem (Unhandled Promise Rejection) occurring on Safari on iOS 11 (working fine on 10) as @RyanK-CloudCover describes. Any workarounds?

[scottschiller]

@RyanK-CloudCover, @kopf: Thanks. Sounds like something has changed with Safari iOS 11 then, where it's become more restrictive than before in regards to the rules for detecting and blocking playback triggered from a user action.

My understanding is onclick() -> setTimeout() has always been blocked, and async / synchronous XHR are likely to also be blocked. iOS has traditionally been pretty restrictive on the click -> play business.

One thing to check: Make sure you're on the latest, or at least a recent version of SM2. There haven't been any real significant updates to handling mobile/iOS for a while, but a recent build should be pretty stable. Current release is 20170601. If you're on something really old, that might be a factor.

You should be able to basically call new Audio('/some.mp3').play() from your existing event handler / stack, and it should work on iOS 11 Safari assuming SM2 is the cause. SM2 should be transparent in this, and ideally you should see the same behaviour between SM2 playback and creating a raw Audio() object. Let me know if creating an audio object instead of calling SM2 works for your case - in which case, I've got some work to do.

Note that the SM2 demos on the homepage have playlist-style behaviour, using the onfinish() event to load and play the next MP3 etc., without issue on iOS 11 Safari from what I've seen. Worth a look. http://schillmania.com/projects/soundmanager2/

They are very simple and make calls to play sound immediately from a click event, so that's likely why they work.

Any chance you folks are doing anything asynchronous before attempting to load and/or play sound? The call to start audio should be immediate and within the same call stack as the user event.

This article is more about <video>, but I suspect the rules are similar for <audio>. https://webkit.org/blog/6784/new-video-policies-for-ios/

[lisongx]

@scottschiller

onerror: function(e) { if (e.notAllowed) { ... } { else if (e.notSupported) { ... } };

This actually looks very clean to me !

For the record, I have been advising against people using auto-play for years. Mobile Safari and others have been blocking it for a long time, and now desktop is just catching up because it is equally annoying. ;) http://www.schillmania.com/projects/soundmanager2/doc/technotes/#mobile-device-limitations

Yes true, it's annoying some time. And after some research I found something interesting in the new safari. They have a kind of "whitelist", which big video/music sites like "Youtube" 's auto-play are allowded. But for other less known product the default is just didn't play.

[scottschiller]

Boo. 👻 I've had some time to poke around this a little more.

With desktop Safari, Chrome and others starting to block playback for HTML5 <audio> and <video> "without user action" similar to iOS Safari (and mobile in general), with some special rules and exceptions for major sites like YouTube et al.

This new world is going to be problematic for browser-based games and the like which have been legitimately using unblocked Audio() playback for years.

Here's the Webkit announcement. Note they say users can disable auto-play entirely, even for silent videos. https://webkit.org/blog/7734/auto-play-policy-changes-for-macos/

Here's the Chromium blog note; Chrome 64 is where the fun begins, in January 2018. https://blog.chromium.org/2017/09/unified-autoplay.html

Chrome looks to be striking a balance similar to Safari, where <video> can be allowed to auto-play, albeit silently. This is one bone they have to throw to developers, with the hopes that people will use MP4 and OGG instead of .GIF. It's much easier on bandwidth, RAM and CPU vs. the current state of affairs, as I'm sure you know; developers have resorted to animated .GIF - basically, the worst "video" format ever 😂, but one which auto-plays on mobile... so, it has been exploited all over. Even I did it myself on the SM2 homepage for a turntable clip I wanted to have work.

(~420 KB after lots of tricks; not bad for 960 x 260, 32 colors and 22fps, if I may.) http://www.schillmania.com/projects/soundmanager2/demo/_image/turntable-loop-960-22f-32c-03.gif

Games and other apps have legitimate use cases to play audio without interaction, but it's understandable that (ab-)use of HTML5 audio/video is all too common and the reason for blocking is well-intentioned; it's to stop auto-playing ads, silent audio trickery and other shenanigans used by ad networks to prevent their code from being throttled in the background.

An example of an "innocent victim", though: My "Survivor" Commodore 64 game remake has broken in desktop Safari 11 due to the intro / loader sequence, which plays a sound and uses whileplaying() in order to show the loading sequence and eventually start the game.

My approach at the time (2011) was OK, as the game would proceed fine if audio support didn't exist at all - it detected and handled that case. Now it's a little more nuanced, since audio restrictions are kicking in.

I see the usual suspect in the console, and then the game doesn't start. 😞 http://www.schillmania.com/survivor/

Unhandled Promise Rejection: NotAllowedError (DOM Exception 35): The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.

Things work fine with sound disabled. http://www.schillmania.com/survivor/?mute=1

Skipping the intro, you'll see Safari complain regularly each time the "heartbeat" sound wants to play in the game, or when things blow up, etc. http://www.schillmania.com/survivor/?noIntro=1

It's also worth mentioning, Safari still seems to suffer from a notable performance issue when attempting to play back a handful of Audio() instances at once. I'm not sure there is a way to disable the restrictions, despite seeing Debug -> Media Flags in Safari's developer / debug menu and trying "Audio needs user action" - seems like play is restricted even though that menu item is not checked/active.

I flagged this issue in 2013 and it was reported as fixed, though I noted Safari 10 still suffered in terms of performance before the auto-play restrictions were added. http://isflashdeadyet.com/tests/html5-audio-timing/ https://bugs.webkit.org/show_bug.cgi?id=116145

Unfortunately, I'm going to have to recommend a more guarded approach to playing audio on desktop. Developers will need to consider that audio may not always load, play and fire whileplaying() and friends, and that they can start to expect more error / exception cases. Ideally, sounds should fall through and should not block the UI or experience as in my unfortunate game example, there. 😉

At the least, folks will need to expect onerror() or similar to fire when playback is blocked, and take that into consideration. In my case, I'll need to have onerror() continue the sequence as though sound is working. The game should be fully playable without audio.

Another thought: I don't want to have SM2 do this by default at start time, but an auto-play block / exception handler and detection might not be a bad idea. Some developers might not want to deal with handling blocks, and just disable or skip audio support entirely if so. Like in my game above, if it's not feasible to jump through the hoops to get audio to play without interaction, I'd probably want to just disable it until if/when there is a proper solution.

It'd be easy enough to attempt to load and play a silent sound, try ... catch any exceptions and look for a Promise() returned from the play() attempt. SM2 could have a setup option to include requireAutoPlaySupport or similar, and perform this test before firing its unready() callbacks, and also as part of the soundManager.ok() test. 🤔

Google's auto-play detection example for <video> seems reasonable. https://sites.google.com/a/chromium.org/dev/audio-video/autoplay

var promise = document.querySelector('video').play();

if (promise !== undefined) {

  promise.then(_ => {

    // Autoplay started!

  }).catch(error => {

    // Autoplay was prevented.

    // Show a "Play" button so that user can start playback.

  });

}

Perhaps the Web Audio API has different rules and/or restrictions - I haven't poked around there yet. I wager that it will be similar. If auto-play were allowed there, you can bet advertisers would be the first to move to exploit it. 🔥

FWIW, it does look like Safari lets audio play without restrictions when loading off the local file system, e.g., my game loading from the desktop via file://.

[kopf]

Hi,

according to the OP in this thread (reddit, I know, not the best source), AudioBuffers aren't hit by this restriction. Does SoundManager2 provide an API to use AudioBuffer objects as opposed to strict Audio objects?

The question is probably completely stupid - in that case I apologise. I'm afraid I'm not a frontend/js developer, have no experience with SoundManager2, but have inherited a codebase with this frontend code and now have to rush to at least find a temporary workaround for this bug until we have time to rework the frontend code :/

[asnoba]

Hi @scottschiller,

I'm not sure if you're aware of this, but when testing this demo http://www.schillmania.com/projects/soundmanager2/demo/api/#create-play-onfinish in Safari 11 I get the same error:

Unhandled Promise Rejection: NotAllowedError (DOM Exception 35): The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.

So I guess there's no way to play through a sequence of audio files in Safari and iOS at the moment?

[scottschiller]

@kopf: Thanks for the AudioBuffer detail. SM2 does not use any of the newer Web Audio API stuff, I've been waiting for quite a while for those APIs to get to a point where they're supported consistently enough between desktop and mobile devices. It's pretty close now, but I suspect playback restrictions are going to throw another wrench in the gears (whether HTML5 <audio> or Web Audio API.) Notwithstanding, it's worth reviewing the state of that stuff between desktop and mobile to see what support is like.

I would not be surprised if Safari starts to lock down AudioBuffer, if it presently allows auto-play or similar. Whatever advertisers can do to work around these restrictions, unfortunately, they are also going to be trying. It may come down to "heuristics" similar to ad blockers, where browsers may decide to allow auto-play based on domain or ???. Who knows. 🤷‍♂️

@asnoba: Thanks for the demo note re: desktop Safari 11. Sounds like it's the same pattern as the current thread, here, with the promise being rejected. This might be due to the onfinish() creating a new sound object, then attempting to load it. Perhaps reusing the same sound object, and/or simply attempting to play() with a new URL may be the way to get this working.

If you use onfinish() to call play() with a new URL, I would expect things to work. If you use onfinish() to create a new sound and call play() immediately, I would also expect that to work.

Interestingly (and thankfully), the demos on the SM2 homepage that have playlist-style behaviour appear to be OK and proceed from one to the next without issue. On desktop, the SM2 UI demos follow a pattern of onfinish() -> createSound() and then play().

It may be that you can't have onfinish() -> load() -> play() these days, you only get one "action" for free without a user click or other interaction.

Because of prior mobile restrictions, SM2 defaults to only having one global Audio() instance. It used to be that only one sound was allowed to play at a time on iOS devices, so it was simplest to allow only one Audio() instance even if SM2 created multiple SMSound instances. Desktops don't have this restriction, and hopefully won't need it. 😬

[elschaefo]

hello all... thanks for the info, this has been extremely helpful for me ! @scottschiller I can verify that this is working for me :

(as scott suggested) Perhaps reusing the same sound object, and/or simply attempting to play() with a new URL may be the way to get this working.

I actually didn't realize that you could just play an existing sound with a new URL. Not only did this work it simplified my code...

in my case (for a community radio station)... I'm trying to play an underwriting ad prior to playing the stream. Sometimes there is a pre-roll, and sometimes there isn't. Furthermore, I had some logic to test and see if a user had already been subjected to the pre-roll within the last X number of minutes so we dont bug the crap out of people.

I had a function that ran out to figure all of that stuff above out, and then based on whether or not there was a pre-roll and it was less than X number of minutes, play the ad. Naturally, Safari's new restrictions didnt like that at all. I guess because of the new restrictions my lookup function was "a step too far" from the user's click to start the stream ?

Anyways.... In my code, I simply got rid of a second (new) sound that I had created for the pre-roll (advert / underwriting) and just re-used the existing sound (var soundID in my code). In other words, previously I created a sound for the main stream, then a second sound for the advert, and tried to play one after the other. That was what broke in Safari 11 (and worked but threw errors in iOS).

therefore ... here is what worked (some generalized code). In the example below, Im trying to illustrate that instead of creating a new sound, I'm just playing an existing sound but with a new URL. :

soundManager.play(soundID,{
    url: audioAssetUrl,
     onload: function(bSuccess) {
     // if we cannot load the advert, play the main stream
      if (!bSuccess) {
          playStream();
      }
      if (bSuccess) {
          // log impression
          setUnderwritingListen()
      }
      soundManager._writeDebug('underwriting '+(bSuccess?'loaded!': 'did NOT load.'));
    },
    onerror : function() {
      //  this on error doesn't seem to work, unfort.
       playStream();
    },
    // this on finish does work !
    onfinish: function() {
           // now that the ad has played, tune into the stream
       playStream();
    }
});

TL;DR : if you keep the sound Object the same, and just change the URL, it works ! In my example above, the onfinish event fires playStream(), which runs another soundManager.play (on the same sound object)... and that is just fine with Safari.

hopefully this convoluted explanation helps someone. Thanks to everyone especially @scottschiller for creating and assisting with SoundManager2 !

[maddenf]

Hi @scottschiller. Are there any updates on this? I noticed there haven't been any updates to the code for quite awhile. Is there a more direct way I can get in touch with you?

[scottschiller]

@maddenf Apologies, I haven't pushed updates to SM2 for about a year at this point. 😬 That said, I'm not retiring from working on the project or anything!

I have an HTML5-only branch in the works, and have been debating how to release it. I think I'll push it as a regular update, and instruct folks who need to retain the Flash stuff to stay on a legacy version.

In addition, I think the removal of the Flash cruft will help simplify the code and then I can also fix up some of the most egregious HTML5 playback issues. Most of them will focus around auto-play and playlist-style behaviour on mobile. I would also like to work in the promise stuff, so SM2 users can be notified if/when auto-play is blocked and handle the promise accordingly.

Finally, there is the Web Audio API which may also be of interest to some folks. From what I've seen, desktop Safari still starts to chug when playing a few Audio() sounds in quick succession.

[scottschiller]

FWIW, the latest HTML5-only branch and updates are yonder. https://github.com/scottschiller/SoundManager2/commits/sm2-html5only

I think to give folks advance notice, it would be wise to push an interim SM2 Flash/HTML5 release that warns users "this will be the last release supporting Flash", or similar, and then the next one can be 100% HTML5.

All major browsers these days have HTML5 support for MP3 if not MP4, and OGG can always be used as a fallback. There is very little need for JS + Flash to get MP3, and IE 9+ support HTML5.

I can effectively move the HTML5 + Flash version on a separate branch and provide minimal updates there, and focus on updating the 100% HTML5 version on master.

[maddenf]

Scott, this sounds great! Thanks for the update and ill checkout the html5 Only code.

On Sat, Jul 7, 2018, 12:13 PM Scott Schiller notifications@github.com wrote:

FWIW, the latest HTML5-only branch and updates are yonder. https://github.com/scottschiller/SoundManager2/commits/sm2-html5only

I think to give folks advance notice, it would be wise to push an interim SM2 Flash/HTML5 release that warns users "this will be the last release supporting Flash", or similar, and then the next one can be 100% HTML5.

All major browsers these days have HTML5 support for MP3 if not MP4, and OGG can always be used as a fallback. There is very little need for JS + Flash to get MP3, and IE 9+ support HTML5.

I can effectively move the HTML5 + Flash version on a separate branch and provide minimal updates there, and focus on updating the 100% HTML5 version on master.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/scottschiller/SoundManager2/issues/178#issuecomment-403237156, or mute the thread https://github.com/notifications/unsubscribe-auth/AKRvoGwlZXiwycJHp_wAprVqdKoIb2W9ks5uEQhPgaJpZM4Pe5Aq .

[evoyy]

My method to resolve this issue is to create a web audio MediaElementSourceNode from the

// sm2 attaches the audio element as sound._a
let audio = sound._a;
audio.crossOrigin = 'anonymous';
sound._sourceNode = audioContext.createMediaElementSource(audio);
sound._sourceNode.connect(inputNode);

function play() {
    /**
     * If an AudioContext is created prior to the document receiving a
     * user gesture, it will be created in the "suspended" state, and
     * you will need to call resume() after a user gesture is
     * received.
     */
    audioContext.resume().then(() => {
        sound.play();
    });
}

@scottschiller

It's great to hear you are working on an HTML5-only version! Good luck with it!

[krue-doug]

I'm running into the same issue, and thanks @scottschiller for the great work!

Any chance you folks are doing anything asynchronous before attempting to load and/or play sound? The call to start audio should be immediate and within the same call stack as the user event.

I am using react-sound which uses this library and playing a notification sound when I receive websocket data.

I think to give folks advance notice, it would be wise to push an interim SM2 Flash/HTML5 release that warns users "this will be the last release supporting Flash", or similar, and then the next one can be 100% HTML5.

If it's a breaking change and there is a deprecation, then a major version update should sufficiently communicate that and a changelog.

[thylle]

iOS Support: Automatic playing of videos is supported as of iOS 10+, but requires the playsinline attribute on the

[shanekoss]

FWIW, in Ember, I got this to work by playing a blank sound for 50ms and then playing the sound in question like so:

        var audio = new Audio("/assets/audio/silencio.mp3");
        audio.play();
        Ember.run.later(
          this,
          function() {
            if (!this.isDestroyed && !this.isDestroying) {
              audio.pause();
              //now play the SM2 sound
              this.get("loadedsound").sound.play();
            }
          },
          50
        );

I was only having the issue when the first item was played - so only needed to do this on the first attempt to play a sound.

[robrtrix]

Hitting the same issue and this worked, and it's super simple:

I'm using this simple audio file, of course, I'd copy it locally and source off your own site. Then I have a my function call that gets a secured URL from private blob storage and returns the Url: getPlayUrl(compositionId, mediaFileTypeId) That's running out of scope of the user click event, and takes a moment to retrieve, and by pausing the silent audio, switching the URL, and clicking play again, it seems to be working great on my iPhone.

   // For iOS mobile to work, we start by playing a silent song
    playUrl = "https://github.com/anars/blank-audio/raw/master/1-minute-of-silence.mp3";
    simpleAudioPlayer = soundManager.createSound({
        url: playUrl
    });
    simpleAudioPlayer.play();

    // Then once we have retrieved the URL a short moment later, we pause the song, set the url and play again and it works!
    $.when(getPlayUrl(compositionId, mediaFileTypeId)).then(function (data) {
        simpleAudioPlayer.pause();
        simpleAudioPlayer.url = data;
        simpleAudioPlayer.play();
    }).fail(HandleError);

[duyhachix]

• Hi, I met this issue when call play() function in this code. audio could play with Macbook, but when i used Ipad, audio.play() not working and log the error 'Unhandled Promise Rejection: NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.'

This is my vue code: ` /**

• Camera button click handler */ async onCaptureButtonClick() { if (this.auto) { while (this.auto) { if (this.sliderPoint === 0) { this.capture(); return; } await this.execute(); } } else { await this.capture(); } },

  async execute() {
      this.countdown = this.sliderPoint;
  
      for (let index = this.countdown; index > 0; index--) {
          this.countdown -= 1;
          await this.sleep(1000);
      }
      await this.capture();
  },
  
  async capture() {
      let camera = this.$refs.camera;
      let audio = new Audio(
          require('@/constants/sounds/capture_sound.mp3')
      );
  
      camera.pause();
      audio.play();
      this.setSpinner(true);

  `