Investigate using libsamplerate for audio conversion

darbyjohnston / tlRender

tlRender is an open source library for building playback and review applications for visual effects, film, and animation.

BSD 3-Clause "New" or "Revised" License

193 stars 22 forks source link

Investigate using libsamplerate for audio conversion #26

Open darbyjohnston opened 2 years ago

darbyjohnston commented 2 years ago

Currently we rely on using FFmpeg for converting the audio in files to a format supported by the audio device. This seems to work fairly well, but since FFmpeg is a large dependency it might be nice to investigate alternative approaches like libsamplerate.

darbyjohnston commented 1 year ago

Some additional notes here: https://github.com/OpenTimelineIO/raven/issues/15#issuecomment-1574060741

ggarra13 commented 1 year ago

BTW, mrv2's tlRender branch now supports resampling on FPS (speed) change both forwards and backwards. Let me know if you want me to clean it for a PR.

darbyjohnston commented 1 year ago

Thanks, but I need to spend some time refactoring the audio code before adding any new functionality. Hopefully I will have some time in a few weeks.

darbyjohnston commented 1 year ago

I experimented using libsamplerate for audio conversion but ran into a few issues:

The converted audio had pops (which was likely my fault).
libsamplerate only does frequency conversion. FFmpeg's swresample library does both frequency conversion and channel conversion which is nice.
libsamplerate only works on floating point data. FFmpeg's swresample library works on both floating point and integer data which is nice.

It still might be nice to use libsamplerate since it has a more permissive license and can be staticly linked.