add support for Web Audio API

[melonjs]

As discussed in Ticket #110

With fallback to HTML5 audio if not supported

To be implemented:

• Remove built in audio support detection, utilize the me.audio.init or ext value from resources for what URLs are available.
• Re-implement something like channels.

[parasyte]

@obiot I targeted this ticket for 0.9.8, because I really want audio on Mobile solved! I think this will get us a step closer, but I am reserving some doubt. I don't know if Web Audio API will truly solve all the issues...

[obiot]

@parasyte Agree, honestly I was almost on the point to start it, but I actually prefer to limit the scope of the 0.9.7 to what we already did (or what is in progress). my point is that we already have a lots of changes on this version, and except for a few small other tickets i would like to work on (Like #167 or #170) I think that's it for 0.9.7 (note that does not mean we cannot work on separate branch). And most of all i don't want again to have the next release in 6 month, i feel we are on the right rails right now and i want to stay there :)

Note except if you have very strong will to work on this one, I will do it (maybe we can even start using the "assign" button)

[parasyte]

@obiot Good point on the "assign" button. It would work well for communicating intentions. (I used it once for #103) Agreed on scope and release cycle!

[obiot]

@parasyte

I just found this (was in the last HTML5 newsletter) : https://github.com/goldfire/howler.js

makes me wonder if we should actually spend time re-implementing what these guys have done already, or just use this one instead (it's release under MIT).

what do you think ?

[parasyte]

Looks like a great general purpose audio library! We should probably experiment with it as a means to get our audio capability "modern".

But I think also it should be implemented as a plugin, like with Chipmunk-js. That would mean I don't have to add the extra 10KB overhead for the library if I don't want it. (See: #212)

Also, there are some considerations as well; the fade methods are implemented with basic setTimeout. I would rather use the Tween library, so I could do things like changing the easing algorithm, and prevent silly bugs when the game is paused while fading. ;) It looks like there are other uses of setTimeout that could also cause this bug!

The other point is it appears to limit the capability of the Web Audio API. So I can't do fancy things like processing sounds (add an echo in a cave environment? apply a doppler shift for cars zooming by?) It supports 3D positioning, but that's it.

[obiot]

Seems awesome, will definitely try this one in a next game ! https://github.com/meenie/8bit.js

[obiot]

Hi guys,

I would like to relaunch the discussion on this one, and for it to be part of the 1.0.0 (we need a better audio support) release.

I was looking again at howler, which is released under the MIT license and it seems to me that this is a very good candidate to replace the current audio implementation, and well written that it could be fully integrated and yet allowing us to keep the current me.audio api

Check the homepage from your mobile : http://goldfirestudios.com/blog/104/howler.js-Modern-Web-Audio-Javascript-Library

[agmcleod]

Nice & simple looking API. Shouldn't be too difficult for us to implement & test out.

[obiot]

so Howler : as the default audio API in meloinjs, go-no-go ?

[agmcleod]

I say go :)

[obiot]

that 2 votes, including the one in the forum !

any volunteer for the task ? :)

[agmcleod]

What the hey, ill add it to my list :)

[petarov]

I did a plugin based Howler.js impl. here as part of the Psiral game we did for Game Off 2013. It is far from finished and I couldn't figure out how to adapt the playTrack, resumeTrack, etc. functions but the plugin based integration concept worked really fine and one may use Howler via the existing me.audio interface and resources.

[obiot]

just tested the howler audio branch from home on my laptop and it's really working like a charm ! I can even notice improvements on the audio support in the browser: sound clips are instantly played (where we had a very small delay before) and one audio clip can now be played multiple times without any issues (sometimes some sounds were skipped before).

it's also fixing the long standing issue #58 with the looping bug, and although i did not really tested it maybe the #109 one, so both (or at least one), could be closed one this one will be fully merged.

I must say i'm very very happy about this one, thank you so much for this !

Aaron, i'm ok to merge it now if you want to push the merge button yourself now, or if you prefer first to take in account the few comments by Jason (if you have the time, else I can look at it as well), as you wish, you are the boss :P

[obiot]

out of curiosity I was searching the net for an Web Audio Polyfill, and found the following one with some black magic for iOS.

https://github.com/shinnn/AudioContext-Polyfill/blob/master/audiocontext-polyfill.js

it cannot really be used as such, but this certainly the clue to fully enable it automatically :P

[obiot]

One question : with howler now in the place what's the point of keeping audio capabilities detection ? https://github.com/melonjs/melonJS/blob/1.0.0-dev/src/audio/audio.js#L155

as highlighted (i don't remember where) by Jason, since Howler will choose the supported one among the different ones we feed him with, we could just use the me.audio.init string as an indication of the format we do provide (if we do not want to change the asset definition too much), or just completely remove it if we do change the asset definition format.

for the latter we could even do something simple by adding a ext field to it that would define the format extension we do provide.

so before :

    {name: "cling", type: "audio", src: "data/sfx/"},

after :

    {name: "cling", type: "audio", src: "data/sfx/", ext:"mp3,ogg"},

[agmcleod]

Not really a fan of the ext type, since it requires a lot of duplicate in the resources array. Technically i don't think there's a reason why we can't do it. I'm just lazy when it comes to declaring my audio i guess ;).

But yes, this is something I agree needs to be done. I think Jason mentioned it before as well.

[parasyte]

I like the idea of me.audio keeping track of the extension list given to its init method. Combining the resource path with the extension list is super easy:

var soundclip = new Howl({
    urls : audioFormat.map(function (ext) {
        return sound.src + sound.name + "." + ext + me.loader.nocache;
    }),
    // ...
});

[agmcleod]

+1. I think that's probably the most straight forward implementation.

[obiot]

@parasyte by the way, did you notice that howler has a pos3d function ? :)

[parasyte]

@obiot Yep.

I didn't see anything useful for dynamic audio in the public API, but I'm sure it's possible to get access to the underlying Web Audio API for that.

[agmcleod]

Experimenting with it a bit, the playing seems to run separate audio instances, which is nice. However, howler does share callbacks. Hmm.

[obiot]

a couple of things I noticed as well :

• why don't we use the onend function for the callback (instead of on/off) onend: Function (function(){} by default) Fire when the sound finishes playing (if it is looping, it'll fire at the end of each loop).
• play does not resume the current audio clip when pause was called first (it instead restart from the beginning)

[parasyte]

@obiot onend was initially used: https://github.com/melonjs/melonJS/commit/cc37f5f35a5e0ba2cdc739749577526f0f328b21#diff-0ee9ee23551e9c51cdda33fcb8540c2dR337

Background: https://github.com/melonjs/melonJS/commit/bccdd30a95c40bf84179450d9dbc8f5563ad7422 and https://github.com/melonjs/melonJS/commit/068d881f3ba9a77a222603c14ef4feda05d68317#commitcomment-5030704

The important discussion has been hidden by deleted branches. :confounded:

[agmcleod]

Ah no wonder i couldnt find them easily. searched my emails on the subject earlier. Typically why I prefer chat on issues or pull requests :P

[obiot]

I guess the synchronous loading stuff can be removed as well now since on mobile web audio will mostly be used and handled by howler ?

shall we also remove me.audio.sound_enable ? Howler should not do anything if no audio is supported, and the same can be done by calling muteAll/unMuteAll from the me.audio.enable/disable function ?

[agmcleod]

@obiot @parasyte think we can close this for now? Perhaps create another issue for further development/exploration into positional audio and other fancy things?

[obiot]

yep, i think we can close it too.

About positional audio, it's there actually (i mean in howler), i guess we then simply need to extend "our wrapper" to also cover those functions.

Also a version 2.0 of Howler should come at some point, so it's maybe also better to wait for it before making any further "big" changes.

[agmcleod]

Yep: https://twitter.com/goldfirestudios/status/427264646098083841

[obiot]

hehe :P

[obiot]

@agmcleod @parasyte

weird, I updated the breakout game with the last 1.0.0 version : https://github.com/melonjs/breakouts and I cannot get the sound to work on mobile devices.... It's working with whack-a-mole, but not with this one, and I cannot see what is different (except maybe the background music)

[obiot]

my bad actually, I forgot that a sound needed to be played ON a click/touch event, it's fixed now ;)

[parasyte]

Yep, web games for mobile devices need an "Enable Sounds" button. It's not an issue with CocoonJS

[obiot]

yep :) it's awesome to finally audio in the browser, I love this new feature in melonJS, thanks to Howler !