search

w3c / webcodecs

WebCodecs is a flexible web API for encoding and decoding audio and video.
https://w3c.github.io/webcodecs/
Other
978 stars 136 forks source link

Is it possible to fetch audio fragments for offline/batch processing? #373

Closed abhinavkulkarni closed 2 years ago

abhinavkulkarni commented 2 years ago

Hi,

I am looking for a browser native solution to do some basic audio processing entirely on the client-side, in-browser. I want users to select a short time range (say, no more than 2mins) anywhere in the audio and post-process the fragment to create short clips with fancy waveform visualization. This is a good example of what we want to achieve.

After spending a good amount of time looking at Web APIs, Webaudio APIs, FFmpeg WASM port (20MB+ in size + licensing issues), I am running into the following problems:

  1. Ideally, once the user has selected the short-time range, we would like to download the audio fragment and add fancy visualization using OfflineAudioContext and custom ScriptProcessorNode.
  2. However, the ability to fetch fragments is not readily available, except through HTML5 audio tag or HTMLElementNode. However, audio tag or HTMLElementNode doesn't allow for batch processing with OfflineAudioContext, your can only process audio in a streaming fashion using AudioContext.

The crux of the problem is that is it not possible to programmatically fetch (based on user input) media fragments with XHR or fetch API. One can use client-side FFmpeg WASM port for this, however large library size disqualifies it for this project.

Please note that the servers serving the audio files do accept Range header (think of most of the podcast mp3 hosts).

Does this project solve the problem of downloading media fragments?

Thanks!

sandersdan commented 2 years ago

Only indirectly. WebCodecs provides audio decoding, but does not provide demuxing. (In FFmpeg terminology, it is like libavcodec, and not like libavformat.)