Feature: audio add-on

[sc0ttj]

Do an audio add-on, so adding fast & responsive audio to components is as easy as possible.

• "fast" - use the Web Audio API, cache the sounds up front, and only fall back to HTML5 audio if need be.
• "responsive" - audio properties can be updated as part of a components state:
  • example: change volume for left/right audio channels as component moves left/right on-screen

..Need to find some examples of good, lightweight (game related) audio libraries that can handle the job.

Example usage would be something like:

import { Component, htmel, useAudio } from '@scottjarvis/component'

// setup a cached audio library
const audio = useAudio({
  ok: 'sounds/ok.mp3',
  back: 'sounds/back.mp3',
})

...which should provide the following:

• audio.ok.fn() - where fn() is one of these: play(), pause(), cancel(), remove(), effect()
• audio.ok.volume = num - set volume for left and right together
• audio.ok.volume = [num, num] - set volume for left and right separately
• etc..

Example usage:

// define a component 
const Foo = new Component({ ... })

// optionally, extract each sound to its own variable
const { ok, cancel } = audio

Foo.view = htmel`
  <div>
    <button onclick="${ok.play}">OK</button>
    <button onclick="${back.play}">Cancel</button>
  </div>`

[sc0ttj]

Component Audio add-ons

• https://github.com/kittykatattack/sound.js - exactly what I want! (by maker of hexi and ga, and pixi.js expert ... 8.6kb .. 4.8 gzipped)
• https://github.com/rezoner/SoundOnDemand - quite small, lots of nice methods to modify sounds (loop, fade, pan)
• https://github.com/madebywild/audioFX - component audio add, can apply dynamic effects, only 2kb :)
• https://github.com/tamasszoke/simple-sfx - almost has the API i want - sfx('filename').play({ volume: 50 })
• https://github.com/wle8300/uifx - see for audio add-on API ideas, nice API
• https://github.com/cyberwani/sfx.js (2kb, nice multi sound setup, good for rapid fire sounds)
• https://github.com/tamasszoke/simple-sfx - same as above? ...very similar API
• https://github.com/goldfire/howler.js/ (45 kb)
• https://github.com/0xfe/vexwarp - audio time shifting and pitch stretching
• https://github.com/lukephills/Sound-Object - creates a simple Sound object, can assign volume and panning

Also see these:

• https://github.com/mmckegg/audio-slot - creates synths by passing in declarative objects
• https://github.com/mmckegg/soundbank - create midi notes, and so on, by passing in declarative objects

The two above have a similar API to Component/useAudio - you pass in nice, descriptive objects, using a setState like method to update or create your sound stuff.

• https://github.com/rserota/wad - very advanced, for MIDI stuff too

[sc0ttj]

See the forked project (https://github.com/sc0ttj/audioFX) for snippets, ideas and features:

• add stereo panning (left/right panning)
• add 3d panning (for 3d "spatial audio")
• audio ducking
• more audio filters
• a setting to toggle automated randomness, for rapid-fire sounds like gunshots, footsteps, etc
• metering (auto-fix clipping)
• choose sound "environment" (use convolver node to create different "room effects")

More reading and examples:

• https://webaudioapi.com/samples/
• http://www.smartjava.org/examples/webaudio-filters/

[sc0ttj]

Do a useAudioUI add-on for the useAudio add-on: which provides auto-generated UI controls for each sound object for easy access to all the settings of the sound object.

Each control would contain the maths required to make the sliders/inputs etc change the settings nicely... for example, to adjust volume you need an exponential scale as 0.5 is not half as loud as 1.0:

if (rotation == -140) gain = 0; else gain = Math.pow(10, (rotation - 140) / 200);

Example:

Create a sound object with the following adjustable settings: volume, pan and lowpass filter.

With the useAudioUI enabled, you would just do mysound.render('.some-container') and you will get HTML based UI added to the page, with inputs and sliders for volume, pan and the low pass filter settings.

When you update the controls, the sound properties will change "on the fly", as the sound is being played.

...Auto generating these UIs for each sound object should make it pretty easy to debug, play with, manage & monitor you applications sounds... It should also be a decent starting point for audio-focused web applications.

[sc0ttj]

Done the main useAudio add-on (not done any audioUI add-ons though..)

See https://github.com/sc0ttj/component/#using-the-useaudio-module