felixpatzelt / colorednoise

Python package to generate Gaussian (1/f)**beta noise (e.g. pink noise)
MIT License
195 stars 20 forks source link

Use np.sum instead of inbuilt sum to improve speed #4

Closed RuABraun closed 5 years ago

RuABraun commented 5 years ago

I was trying to figure out why this package was so slow, and while running the commands line by line noticed that it was using the inbuilt sum instead of numpy.sum. It is pretty straightforward to see what a difference it makes when running the line ix = sum(s_scale < fmin). Using np.sum is much faster.

To match the existing coding style I imported the sum function, but to make it explicit that this is not the inbuilt python sum function I renamed it.

edit: Of course this isn't the bottleneck, but still.

edit: Okay I found out my real issue, for some sample lengths the FFT is extremely slow, it worked fine in my instance to create samples that were a power of 2 and to tile them. Hopefully someone else can learn from my mistake.

felixpatzelt commented 5 years ago

Thanks for finding this! The performance improvement is unfortunately quite esoteric (~10µs per call on my machine). I'm still merging it because it makes sense to use numpy's sum function here.

With respect to the FFT being "slow", the question is relative to what? It is actually a very fast algorithm, but it is most efficient when used on arrays with lengths that are a power of 2 (as you found out already). See e.g. the note in the numpy documentation or this tutorial. Prime number sizes are usually the slowest. There are even faster algorithms than numpy's, but I didn't think this would be enough of an issue to add another dependency.

How are you using this package? The best way is to generate the random numbers in very large batches and not inside a loop. E.g. cn.powerlaw_psd_gaussian(exponent, (N,T)) to create N time series with T time steps.

RuABraun commented 5 years ago

I was generating it for moderately large arrays (up to an 5 minutes of audio at 16khz). It works great now, just needed to make sure I chose the length appropriately (it was basically random before) :+1: . Feel bad about opening up with "trying to figure out why it was so slow" :( but when I realized what my mistake was I had already opened up the PR...