[DeepDive] Web app audio

[0xdevalias]

Web App Audio

• Basic
  • https://developer.mozilla.org/en-US/docs/Web/HTML/Element/audio
  • https://developer.mozilla.org/en-US/docs/Web/API/HTMLAudioElement
  • https://developer.mozilla.org/en-US/docs/Web/API/HTMLAudioElement/Audio
• More interesting/advanced (web audio API's)
  • https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API
  • https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Basic_concepts_behind_Web_Audio_API
  • https://developer.mozilla.org/en-US/docs/Web/API/AudioContext
  • https://developer.mozilla.org/en-US/docs/Web/API/AudioNode
  • https://developer.mozilla.org/en-US/docs/Web/API/StereoPannerNode
  • https://developer.mozilla.org/en-US/docs/Web/API/StereoPannerNode/pan
• I believe we can also stream audio tracks across WebRTC, etc
  • https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/webrtc-integration.html

  • The following examples illustrate WebRTC integration with the Web Audio API. They are borrowed and modified from Robert O'Callahan's MediaStream Processing API proposal.

  • https://developer.mozilla.org/en-US/docs/Web/API/AudioContext/createMediaStreamDestination
  • https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamAudioDestinationNode
  • https://developer.mozilla.org/en-US/docs/Web/API/MediaStream
    • https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API#WebRTC_concepts_and_usage
  • https://webrtc.github.io/samples/src/content/peerconnection/webaudio-input/
  • https://github.com/webrtc/samples/tree/gh-pages/src/content/peerconnection/webaudio-input
• Etc
  • https://dev.to/areknawo/9-libraries-to-kickstart-your-web-audio-stuff-460p
  • Howler.js, Tone.js, SoundJS, etc
• Browser Support

Howler.js

• https://howlerjs.com/
• https://github.com/goldfire/howler.js
  • https://github.com/goldfire/howler.js/tree/master/examples
  • Docs
  • https://github.com/goldfire/howler.js#core
  • https://github.com/goldfire/howler.js#plugin-spatial
  • https://github.com/goldfire/howler.js/#codecsext
  • https://github.com/goldfire/howler.js/#format-recommendations

    • If your goal is to have the best balance of small filesize and high quality, based on extensive production testing, your best bet is to default to webm and fallback to mp3. webm has nearly full browser coverage with a great combination of compression and quality. You'll need the mp3 fallback for Internet Explorer.

  • Loading core/plugins separately

Blogs, etc

• https://medium.com/game-development-stuff/how-to-create-audiosprites-to-use-with-howler-js-beed5d006ac1
  • https://github.com/tonistiigi/audiosprite

React / Redux Wrappers

• https://github.com/E-Kuerschner/useAudioPlayer
• https://github.com/joshwcomeau/redux-sounds
• https://github.com/thangngoc89/react-howler
• https://github.com/morinted/rehowl
• https://github.com/vbalastegui/React-Hook-Howler

Audio Containers, Codecs, etc

• Browser Support
  • https://caniuse.com/#feat=webm
  • https://caniuse.com/#feat=ogg-vorbis
  • https://caniuse.com/#feat=opus
• https://en.wikipedia.org/wiki/WebM
  • https://www.webmproject.org/
  • https://www.webmproject.org/code/
• http://www.opus-codec.org/

  • Opus is a totally open, royalty-free, highly versatile audio codec. Opus is unmatched for interactive speech and music transmission over the Internet, but is also intended for storage and streaming applications.

  • https://github.com/ImagicTheCat/libopusjs

  • a libopus API for JavaScript (wasm/asm.js)

  • https://en.wikipedia.org/wiki/Opus_(audio_format)

  • Opus replaces both Vorbis and Speex for new applications, and several blind listening tests have ranked it higher-quality than any other standard audio format at any given bitrate until transparency is reached, including MP3, AAC, and HE-AAC

  • https://en.wikipedia.org/wiki/Vorbis

Audio Conversion

• https://en.wikipedia.org/wiki/Core_Audio_Format
  • https://ss64.com/osx/afconvert.html
  • https://apple.stackexchange.com/questions/365616/what-are-the-command-line-options-for-afconvert
  • afconvert --file caff --data opus SomeTrack.wav
• https://www.ffmpeg.org/
  • ffmpeg -i SomeTrack.wav -acodec libopus SomeTrack.webm
  • ffmpeg -i MyMusicTrack.wav MyMusicTrack.mp3
• https://gist.github.com/protrolium/e0dbd4bb0f1a396fcb55
  • using ffmpeg to extract audio from video files