Reduce memory consumption by chunking media

[wilsonlmh]

As @katspaugh said, "We do not control how decoding works", I think split input media into pieces will be a good solution to get rid of crashes while loading big files(And keep the peaks only with much much lower resolution. Or decode/cache on demand). However it's a big challenge and huge workloads. I have no experience on de/remuxing any binary format. I hope somebody can involve together. Below is some research:

Here’s native support audio codecs list: https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats

So they are opus, vorbis, mp1/2/3, aac and PCM. If we only deal with these codecs, we need to remux (minimum) 3 types of containers:

opus, vorbis -> ogg mp1/2/3, aac -> mp4/m4a PCM -> wav (tested PCM in ogg doesn’t works)

Ref: https://en.wikipedia.org/wiki/Comparison_of_video_container_formats#Audio_formats_supported

Existing muxer(I mean something we can reuse): ogg -> https://github.com/jpemartins/speex.js (demux and mux) mp4/m4a -> https://github.com/videojs/mux.js (demux and mux) wav -> http://webaudiodemos.appspot.com/AudioRecorder/index.html (seems mux only)

Talk me about what you think?

[wilsonlmh]

To explain my concept:

• Steps of chunking and buffer peaks
  1. input media
  2. demux, determind format and codec
  3. find out location of next chunk inside raw stream
  4. extract chunk(of the raw stream)
  5. mux into suitable format and store in RAM
  6. let WebAudioAPI decode it into buffer
  7. calculate peaks and push into uInt arrays(UInt arrays are fast, and almost 1:1 space)
  8. delete/release the chunk(let browser collect it)
  9. repeat from step (3) until finish processing whole file
  10. return to normal drawing

For your reference, the sample rate/accuracy of wavesurfer.js can provide (base on my test) is about 96Hz. It means wavesurfer draw 96 pixels for each second of audio.(This doesn't relative anything to the sound frequency) For:

• 960Hz/16bit/channel

You can get 65536 levels and 960pixels/second, which detail enough for retina display but only 7MB/hour/channel memory needed.I personally suggest 960Hz/16bit or 960Hz/8bit(sample rate can adjust) will be suitable for this purpose.

[entonbiba]

@wilsonlmh I this would perform better with a hls audio type source, #1078 let's continue the discussion at that ticket instead.