Implement 'frequency scrunching' (averaging across frequency) to improve channel isolation

[telegraphic]

It would be nice to be able to use rawspect to do a longer-than-required FFT, then average neighboring frequency channels together. This improves inter-channel isolation (@david-macmahon and I have confirmed this works!)

For example, if you had 100 MHz bandwidth data, and wanted spectra with 1 MHz channel, 100 us resolution you might do a 100-channel FFT then average in time until you reach 100 us. However, it is actually better to do a much longer, 100000 channel FFT, then average back down to get to MHz channel width.

Better again is to do a polyphase filterbank, but that's more complicated ;)

[radonnachie]

That doesn't seem far off from the implementation for ICS.. of course the kernel will have to accumulate across different indices, but the development isn't that long winded... There are inferred upper limits on FFT sizes too. It'd be nice to have those raised, if not razed.

[david-macmahon]

When producing multiple output products of different resolutions, rawspec currently performs FFTs at each resolution (i.e. length). Because we require all the FFT lengths to be factors of a common overall size (i.e. the LCM of the highest resolution and the number of time samples per block), we could just do one long FFT (e.g. for the highest resolution product) and then channel average the output powers down to the desired lower resolutions (as well as the highest output resolution as is already done). This would reduce memory footprint (fewer FFT output buffers) AND reduce the number of FFTs that need to be done AND provide cleaner channel isolation in the lower resolution products. Win, win, win!

[radonnachie]

That too is a cool reward for this...

So we'd start by developing frequency scrunching capabilities and then we'd develop that multiple FFT lengths merely triggers appropriate frequency scrunching.

I'd be happy to work on this if no-one else has the time...

[david-macmahon]

I think they're kinda one and the same. For the low-res products, we already integrate multiple FFT output spectra into one power spectrum, so the only change is to use a common FFT output buffer (i.e. the FFT output buffer for the highest res output product) and then change which channels get integrated where. Knowing when to dump each power product is maybe a little tricky, and there might be fewer CUDA streams (I forget whether there's one CUDA stream per output product or just one for all products).

[david-macmahon]

What will require (more) extra work is arranging for a long FFT even when only a short FFT is required (e.g. when a single 8 point FFT per coarse channel output product is requested). I think we'll need to create another command line option for this to allow the user to specify the "coherent resolution" (the internal FFT size) which can be equal to or finer than the "incoherent resolution" (i.e. the resolution of the output power spectra).