Reassigned spectrogram for an audio signal

[josephernest]

Congrats for this library!

I'd like to plot the reassigned spectrogram of an audio frequency sweep (from 100hz to 20khz), available for download here as a WAV file.

from scipy.io import wavfile
from tftb.processing import *
import matplotlib.pyplot as plt

sr, x = wavfile.read('test.wav')
x = x[:1000]
s = reassigned_spectrogram(x)
plt.imshow(s[0])
plt.show()

Problem:

• With a length of 1000 samples only (i.e. only 23ms), it already takes nearly one minute to compute! So it's nearly impossible to test this reassigned spectrogram on a 10 seconds audio file. How to improve the performance in this specific case (non-complex real audio signal)? (Some audio editors are able to plot reassigned spectrograms quickly, such as Izotope RX, see here)

• Even on the small portion of audio I tested, I couldn't get a nice plot of the reassigned spectrogram. Would you have a code example of plot of a reassigned spectrogram? Thank you in advance!

Result that I'm looking for:

[jaidevd]

Hello @josephernest

Thanks for your interest. There's a lot that can be done to improve the performance of not only the reassigned spectrogram, but pretty much everything else. The way it stands right now is just a simple, equation-to-code implementation of various TFRs - which I'm sure is suboptimal in many cases.

I'll take a look at this soon, but I'm afraid it'll take me a couple of weeks at least, as it is PyCon season in India :smile: (https://in.pycon.org).

Thanks,

[josephernest]

Oho Pycon, that's great, I'll try to go to one, one day!

Feel free to ping me if you have news about this, it would be interesting to be able to compute and display a reassigned spectrogram like described here :)

[jaidevd]

Hi @josephernest,

I'm doing some basic line-by-line profiling of the reassigned spectrogram code, and so far I've been able to knock off nearly 34% of the runtime for the same example as you've mentioned, for the same sweep.wav file. By default it was taking about 108s on my machine to run, and now I've got it down to ~ 70s. This was caused because I was unnecessarily calling numpy functions on Python lists. Numpy functions do work on lists but I guess they first convert those lists into an array, which is expensive. So, when carrying out numerical operations on Python lists, it's important to consider whether it's worth converting them to arrays.

Anyway, this is still not as fast as it should be. I'll see what else I can do over the next couple of days.

[josephernest]

Great news @jaidevd! 70 seconds of processing time for just 1000 samples of audio (i.e. 23 ms of audio) means it would take ... 85 hours of processing time for a reassigned spectrogram of a 10-second audio file...

Do you think there's something to do allowing several orders of magnitude improvements?

AFAIK some audio editors can display reassigned spectrogram of 10 seconds in 10 seconds of processing time.

[jaidevd]

Do you think there's something to do allowing several orders of magnitude improvements?

Yes, vectorizing the whole operation is what will do that.

[aoliveira-dtx]

Hi. I've noticed that when the signal is a Pandas Series there's an error. Also, when I try to input the time stamps there is also an error, likely because the test for unique diff is not tolerant to unavoidable rounding. I've also a question. Why not use the library fttw? I believe that is faster than Numpy FTT implementation.

Best regards.