Strategies analysis

[dy]

1. MediaRecorder with timeslice for recording, SourceBuffer→ MediaElement for playback

• presumably fast
• presumably cross-browser
• native-ish
• minimum time slice is limited to ~ .3s or so, playing small slices is impossible (or requires stuffing silence somehow)
• no way to adjust frame → bar correspondance
• hard to debug

Actually mb problem was initial chunk - slicing subsequent parts is fine.

1.1 MediaRecorder, but full blob for playback

Workable solution at react-mic uses media recorder with 10ms slices, creates resulting blob from a bunch of chunks, playback has no preview/edit.

• there's minimal chunk limitation, not as strict as SourceBuffer case
• there is discrepancy of timeslice time returned by recorder in regular intervals and actual intervals... Fixable by real timer or full-file playback
• blocks rearrangement is problematic. According to SimpleBlock opus structure, there is a timestamp in each block, which is (possibly) used for ordering purposes, meaning cutting/pasting blocks is not necessarily going to work, at least it is error-prone (wrong). See reading webm opus. Practically blocks rearrangement works fine, but allows smaller block size than FF, therefore FF has first-block distance apart and a bit more problematic for rearrangement.

Ok, this technique is possible for playback vis, but not editing.

2. web-audio-read for recording, web-audio-write for playback

• not standard way to record audio
• any resolution handling
• the most purposeful method to visualize audio
• extensible, less bugs

[dy]

web-audio-read strategies:

• analyzer getFloatTimeDomainData + some scheduler (source buffer, mediarecorder)
• audio-worklet
• mediarecorder with decoder

[dy]

. Manipulation is possible internally only via single AudioBuffer or a list of AudioBuffers (or simplified raw data chunks). . Obtaining these chunks is only (reliably) possible via AudioWorkletNode, from media stream. In fact that's how it's done in audio-recorder-polyfill. Encoder is a worker there. . Should playback be a. streamable web-audio-write (worklet connected to destination context), b. AudioSourceNode (prerendered audioBuffer) or c. Audio (suppose recording as wav)? . We need it downloadable, possibly even downloadable chunk. . We need to be able to copy chunk as blob URL. . We need diverse codecs support, not just Chrome/opus only. . Raw data chunks come for free from just reading audio worklet, and we ought to read it anyways, no even need in recorder. . Only blob is downloadable, is playable for free too . Any AudioNode can be recorded via mediaStreamDestination.

If we support oscillators, editing must be buffers. (Also that gives precise audio manip benefits.) If we support immediate download of edited file, editing must be blobs. (otherwise we're limited to wav encoding) Seems that we need 1. raw chunks (audio buffers) 2. media recorder to encode them small-size

[dy]

A strategy to convert audio buffers to downloadable blob in short time. . Since blobs don't have to be arranged by timestamp, they allow rearrangement . Multiple buffers can be recorder in parallel, and chunks combined then into a single blob

The strategy is workable, here

[dy]

Settled for working with single wav file and simple audio element