spotify / pedalboard

🎛 🔊 A Python library for audio.
https://spotify.github.io/pedalboard
GNU General Public License v3.0
5.15k stars 260 forks source link

Add pedalboard.time_stretch for time stretching. #249

Closed psobot closed 1 year ago

psobot commented 1 year ago

Pedalboard already includes Chris Cannam's excellent Rubber Band library for its PitchShift plugin. This PR adds a tiny utility function, called pedalboard.time_stretch, to expose Rubber Band's other useful bit of functionality: time stretching.

Note that as time stretching changes the length of the audio, this functionality is exposed as a separate standalone function, rather than as a Plugin subclass, and as such cannot be included in Pedalboard effects chains. However, getting time stretching "for free" by adding ~128 lines of glue code and tests seems pretty worthwhile.

f0k commented 8 months ago

@psobot: Thanks a lot for doing this, very useful! In extending some augmentation pipeline, I noticed that the time_stretch() function is slower and gives different results than the PitchShift plugin:

In [1]: import numpy as np

In [2]: import pedalboard

In [3]: x = np.sin(2 * np.pi * 0.001 * np.arange(250*512)).astype(np.float32)

In [4]: %time y1 = pedalboard.time_stretch(x, 44100, pitch_shift_in_semitones=2)
CPU times: user 371 ms, sys: 15.3 ms, total: 386 ms
Wall time: 385 ms

In [5]: %time y2 = pedalboard.Pedalboard([pedalboard.PitchShift(semitones=2)])(x, 44100)
CPU times: user 193 ms, sys: 0 ns, total: 193 ms
Wall time: 193 ms

In [6]: import matplotlib.pyplot as plt

In [7]: plt.plot(y1.squeeze()[:1000])
Out[7]: [<matplotlib.lines.Line2D at 0x7ff160723460>]

In [8]: plt.plot(y2.squeeze()[:1000])
Out[8]: [<matplotlib.lines.Line2D at 0x7ff160738160>]

plot

Off the top of your head, do you know the reason? Setting high_quality=False gives a similar speed to PitchShift, but yet another result (a third one). I'd just like to understand if I'm missing out on anything when switching from PitchShift to time_strech() (for the use case when I actually need both stretching and shifting).

/edit: Ok, PitchShift uses rubberband in real-time mode and primes it with a second of silence, while time_strech() uses rubberband in offline mode, analyzing the full audio buffer before computing the output. This is enough to explain the difference, and I would assume pitch_shift() generally gives better results at the expense of additional processing time.