smu160
commented
2 months ago Hi @abstractqqq,
Thank you for your interest!
With respect to the 2x claim: Another individual already benchmarked memory usage (as well as run-time). You can find the results and repo here. The takeaway is that the 2x figure that we used, and you cited is rather conservative. As you can see from the results, that is certainly an underestimate. Let's consider your use case(s):
Let $N$ be the sequence length, then
at $N = 1024$, PhastFT utilized 16.32 KB, whereas RustFFT utilized 50.04 KB.
at $N = 2048$, PhastFT utilized 32.7 KB, whereas RustFFT utilized 99.2 KB.
Now, if you pad your sequence length of $N = 1025$ to the next power of 2, you would end up having a sequence length of N = 2048. However, in accordance with the figures you see above, you would still be using less memory.
Note that in the general case of arbitrary length sequences, one would need to use Bluestein's algorithm. The caveat is that the CZT (i.e., Bluestein's algo) still requires padding. Moreover, it would not be as efficient.
Nevertheless, padding is not a one-size-fits-all solution, so adding an implementation of the CZT (i.e. Bluestein's algo) is on our roadmap.
Apologies if this is a bit long-winded, but your question is important, and I wanted to make sure I address all aspects of it.
Best, Saveliy
abstractqqq
Hi, I would like to get some clarification on the 2x less memory claim. In my project I constantly deal with long series of data. If I have to pad with 0s, then memory usage will be higher.
I wonder if the 2x claim is still true when I need to pad? E.g in an extreme case I have a series of 1025 length. Does that mean I need to pad it to 2048? Would memory usage be the same in that case? Thank you.