search

drewcook / arbor-ui

A collaborative, music-making NFT platform for artists and collectors, built on Ethereum.
https://arbor.audio
GNU General Public License v3.0
7 stars 7 forks source link

Improve audio merging from the backend service #190

Open drewcook opened 1 year ago

drewcook commented 1 year ago

Enhancement

User Story

Currently audio merging is handled in a flattening service over https, this requires the user to upload the individual file blobs both to the flattening service and to IPFS during the metadata creation.

In order to remove the need for multiple uploads, we should handle the audio file muxing in browser.

Where we are currently sending the audio blob to the backend merging service, instead we should use ffmpeg to handle merging of multiple file types. This should also enable us to use all of the filetypes supported by wavesurfer. Once the merging is done, the user will still need to upload the flattened audio file to IPFS.

References

noahdahlman commented 1 year ago

ffmpeg.wasm is a pure WebAssembly / JavaScript port of FFmpeg. It enables video & audio record, convert and stream right inside browsers. We should try to use this lib to do in browser audio merging.

drewcook commented 1 year ago

The merging logic with ffmpeg would probably be best to be pulled out in a separate file as a utility and then imported into the component file for use, for separation of concerns and to be able to reuse it later, or create a suite of ffmpeg utilities.

noahdahlman commented 1 year ago

This is an example of doing concatenation. See also this , which demonstrates the flags for overlaying audio.