search

OpenTimelineIO / raven

Raven - OpenTimelineIO Viewer Application
Apache License 2.0
83 stars 22 forks source link

Audio waveforms #15

Closed jminor closed 8 months ago

jminor commented 1 year ago

Most NLEs show audio waveform shapes in audio clips on the timeline. Raven would be greatly improved if it had this feature also.

Here's a mockup of what that could look like

image

Adding such a feature, however, opens many questions about media file formats, loading of media reference target_urls, and other specifics which might be problematic. By design Raven doesn't show video, or play video or audio, so showing waveforms invites the question if the tool should go in that direction or not, which is... complicated.

darbyjohnston commented 1 year ago

tlRender will have support for audio waveforms soon, maybe that could help? :)

Other complications include handling multiple audio channels, bit rates, and sampling. Also handling audio files in .otioz files is another level of complexity.

meshula commented 1 year ago

A friend and I wrote this some years ago.

https://github.com/ddiakopoulos/libnyquist

It's very robust, and is used for desktop based webaudio, and also in a bunch of headsets; so it's known to be bug free and robust. It loads pretty much everything, and solves all of the vexing build issues certain audio format libraries present.

We use libsamplerate for high quality and performant resampling, and miniaudio for easy crossplatform integration. I evaluated all the libraries I could find, and wrote my own reference implementation, libsamplerate was by far the best in terms of ease of use and quality of results and speed.

miniaudio works for playback on ... everything.

[submodule "third_party/libnyquist"] path = third_party/libnyquist url = https://github.com/ddiakopoulos/libnyquist.git [submodule "third_party/libsamplerate"] path = third_party/libsamplerate url = https://github.com/libsndfile/libsamplerate.git [submodule "third_party/miniaudio"] path = third_party/miniaudio url = https://github.com/mackron/miniaudio.git

darbyjohnston commented 1 year ago

That looks quite nice, I’m currently using the FFmpeg swresample library which works well but is a heavy dependency.

jminor commented 1 year ago

Once this feature lands in tlRender, maybe we can look into synchronizing the playhead between tlRender and Raven instead of having redundant features in both...

jminor commented 8 months ago

Thinking about this more, I feel that we should avoid reading media in Raven, and rely on other tools for features like this, video, etc.