Audio on mobile

[adam-singer]

I have not had any success with playing audio on iPhone 5 Safari/Chrome or Android Chrome.

I've used the following

SoundMixer.engine = "AudioElement";

and

  resourceManager.addSound("Sound1","sounds/Sound1.wav",
      new SoundLoadOptions(mp3: true, ogg: true, wav: true, ignoreErrors: false));

Audio works great on normal browsers.

[bp74]

Hi Adam,

I'm not sure about Android but the issue on iOS (Safari) is that they don't allow to play any audio without user interaction. You have to click on a button and only then you can start the playback of an audio element. Of course this is pretty much useless for a game. Therefore the audio engine on older versions of iOS is set to a dummy engine which does not hang/crash but does not play any audio. But the last version of iOS has support for WebAudio and StageXL should auto detect which audio engine is best suited.

I don't have access to an iPhone 5 but i tested it with an iPhone 4S and it worked fine. It would be great if you could check the code in "SoundMixer.dart" and the reason why the AudioContext for WebAudio isn't detected correctly. Maybe it is even a bug in Dart because i use the "AudioContext.supported" property provided by dart:web_audio.

[bp74]

Btw. the "SoundMixer.engine" property is not meant to be set, it should be a getter only :) The library should auto detect the best matching engine for your device. So this is a bug for sure.

[adam-singer]

Looking into it, I remember why I set SoundMixer.engine = "AudioElement";. If I do not set it then Dartium will crash with an "Aw, Snap". Have you noticed this or should I try making a minimal sample of it and submit to dartbug.com.

[adam-singer]

Ok, after some limited testing on mobile. Sounds could be played if the request came from within a event handler such as "onTouch".

  html.document.body.onTouchStart.listen((e) {
    resourceManager.getSound("Sound1").play();    
    });

This worked for iPhone5 Safari/Chrome and Android Chrome. I guess the question is how to design around this? Is it possible to manually dispatch a dom events? Maybe we can listen to that event and play sound from the handler listening on it.

[bp74]

I saw the "Aw, Snap" crash in Dartium before but most of the time those crashes disappeared after a few revisions. Probably you see the crash right when you create a new instance of "new AudioContext".

The workaround with a manually dispached dom event sounds good and it is worth a try, but i am not very optimistic that it will work. This workaround has to be added to the AudioElementSound.load method too because the browser doesn't even start to download the audio sample without user interaction :(

[adam-singer]

yep, manual dispatch event did not work. Listening to on document.body.onTouchStart is one hackish way to be able to call audio play(). But you have to know which elements need to be played and what happens if onTouch fires before you get to queue the sound to play. Have you had any possible work arounds in this area?

[bp74]

I've searched for all kinds of workarounds on the web but nobody had one. Everyone was complaining about the stupid implementation of the Audio element in Safari. The only workaround was to use a mock audio engine which behaves like the other ones but doesn't play any sounds.

When iOS6 was released i tested it with the Web Audio API and it worked fine. But i saw a comment somewhere that Web Audio API got the same restriction like Audio element now. You have to play the first sound in an event to "enable" the audio playback on the web site - but i had no chance to try it by myself.

[nilsdoehring]

Wanted to let you know that "Aw, Snap" occurs even on a fresh download (today) of Dart Editor / Chromium (OSX) in the stock example04. Actually, just doing an addSound to resource manager is triggering the error.

Output of Dart Editor's console:

StageXL: supported audio types are: [ogg, wav]
StageXL: supported audio engine is: WebAudioApi

financeCoding's line 'fixes' this.

SoundMixer.engine = "AudioElement";

[bp74]

Thanks for your feedback. I tested it myself and i don't see the "Aw, Snap" in Dartium. Windows 7, 64 Bit, Dart Editor version 0.1.2_r23608 (latest available build).

Do you use a Mac or Linux? Here is a little test app you can try: http://www.stagexl.org/temp/test1.zip

If you get the "Aw, Snap" error with this app it would be great if you could post it on www.dartbug.com

[adam-singer]

I'm working on MacOSX

Darwin Adams-MacBook-Air.local 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64

Dart Editor version 0.5.11_r23200
Dart SDK version 0.5.11.1_r23200

I see an update has happened, I will retry with the latest. I'll try and remember to file a detailed bug tonight.

[nilsdoehring]

@financeCoding i think this issue applied to both our cases, as the url does indeed work in chrome: https://code.google.com/p/dart/issues/detail?id=9318

the 23552 update resolves this.

@bp74 thank you for your fast help – keep up the good work. like it very much and look forward if/what the flash cc exporter adds to it. do you happen to know if it will run via command line (i.e. ant), also?

[adam-singer]

@bp74 this is the best solution I've come up with, its a hack of a solution but works. I'm thinking of using this in some startup modal before loading the game. It seems you only need to play a sound once and then all other sounds are available to be played.

  StreamSubscription singleShotStreamSubscription;
  singleShot(e) {
      print("playing DummySound from onTouchStart");
      resourceManager.getSound("DummySound").play();
      singleShotStreamSubscription.cancel();
  };

  singleShotStreamSubscription = html.document.body.onTouchStart.listen(singleShot);

[bp74]

Okay this is very interesting and should be a part of the library somehow, i will think about it. Does it also apply to the WebAudio API on iOS6 or only to the Audio element?

[adam-singer]

@bp74 I had SoundMixer.engine = "WebAudioApi"; set for the example above, I'll check now if the same is true for AudioElement

[adam-singer]

Not having great results with AudioElement, hitting error messages Loading resouce failed: Sound.DummySound - Bad state: Failed to load audio. and when it does actually load the sounds play but are clip if I have consecutive sounds to play. I think originally I was forcing AudioElement cause I did not have the right type of sound files, now I think WebAudioApi is producing better results with the hack above. Also that hack should probably live somewhere after all sound resources have been successfully loaded.

[adam-singer]

ugh, so Uncaught Unsupported operation: This browser does not support Web Audio API. happens on Android devices with Chrome... iPhone device works great!

[elsassph]

This single-shot dummy sound should be played (automatically) on iOS6+ only

• we don't know what will be the rule (if any) for Chrome Android when it hopefully will support this API.

This trick should be documented though.

On Sun, Jun 9, 2013 at 6:07 PM, adam notifications@github.com wrote:

ugh, so Uncaught Unsupported operation: This browser does not support Web Audio API. happens on Android devices with Chrome... iPhone device works great!

— Reply to this email directly or view it on GitHubhttps://github.com/bp74/StageXL/issues/22#issuecomment-19168591 .

Philippe

[adam-singer]

funny enough that caniuse.com says chrome 27 should be supported, but android version of chrome 27 does not support it.