Improve/fix signal aliases causing "ghosts" during decimation (zoom)

[dmitryelj]

There are 2 cases, I've found:

• top-left window - at the edge of the screen sometimes appears the signals, that are disappeared when user scroll them to the center or tap on them
• main spectrum window - when I zoom in (with mouse wheel) at some zoom level appear new stations that are really not exist (when clicking on them no sound and nothing at the top-left view).

Mac OS, receiver: SDRPlay, S/R: 2M (sometimes 5M), frequencies: 100-300KHz.

[cjcliffe]

@dmitryelj CubicSDR utilizes decimation on the main waterfall which can introduce aliases -- it's not really a bug more of an aliasing / optimization artefact which I may be able to reduce in the future with some filtering. Changing the center frequency while zoomed can sometimes knock them out of the way or just zoom in/out a bit more until they flatten out.

There's a similar known issue; if you go hunting in the empty spaces between signals (especially in the WBFM bands) the polyphase filterbank channelizer will sometimes create signal aliases you can actually hear that aren't visible on the main waterfall but will show in the top-left view. Those signals are just wrapped from shifting the baseband of the channel the signal falls into -- I posted a detailed description about the issue here: #295

[dmitryelj]

I've attached 2 screenshots. 1) Top-left screen (phantom station really 30-40KHz lower) 2) Main spectrum (zoom phantoms)

Thats really annoying because this phantom signals have big levels, and it's hard to differentiate them from real ones, I need to play with frequency and zoom to determine if they disappear or not.

[cjcliffe]

@dmitryelj I agree it's a bit annoying and I do hope to improve it so I'll mark this down as an enhancement to look at for 0.2.x -- I know there's more I can do with filtering and improving the decimation transitions at least.

[vsonnier]

@cjcliffe Hello ! Following the issue #556 where the ghost/aliasing issues are poping up like the real bothering ones for otherwise enthusiastic users, I re-read the whole set of issues and started experimenting from my understanding of them. As a result, I propose the following user-selectable 'performance' levels for those who want to burn more CPU:

Perf. level	Channelizer size	internal FFT size	Remarks
0	CHANNELIZER_RATE_MAX	DEFAULT_FFT_SIZE	Normal setting today.
1	CHANNELIZER_RATE_MAX	DEFAULT_FFT_SIZE * 4	Tested simply by changing the DEFAULT_FFT_SIZE constant, so both internal and display resolution changes ! But the idea here is just to change the internal resolution, and use decimation / interpolation on the display part. Indeed on a reduced window when the OpenGL display is not refreshed, you can see that compared to the channelizer size change / removal, this eats little CPU. Pros: less main waterfall aliases ?
2	CHANNELIZER_RATE_MAX * 2	DEFAULT_FFT_SIZE *4	Doubles channelizer size. Pros: less chance to be close to an edge for a demod, so potentially less ghosts ?
3	N/A, use runSingleCH() all the time.	DEFAULT_FFT_SIZE * 8	Tested on 2.4MHz RTL-SDR, needs 2x more CPU for multiple demods scenarios compared to perf. 0. Still perfectly manageable on my aging computer. Pros: no ghosts ( apparently ?), even less aliasing on main waterfall. Also reducing the number of demods reduces the CPU if computer can't keep up.

So @cjcliffe I would appreciate some advice here. If you agree, I intend to develop this feature on my own, so you can concentrate on the serious stuff :)

[vsonnier]

@cjcliffe Now the solution above is probably too brutal. What of the 60Mhz BW recent devices ? Pushing that amount per-demod is likely to be too much, even for the fastest CPUs.

So lets me expose a kind-of solution using not only 1 channelizer, but several ones dynamically created to fit the existing demods the best way they can. Note:

• max_channelizer_rank = int(sample_rate / CHANNELIZER_RATE_MAX) the standard number of channels as we know,
• channelizer(k) = {channelizer_bw(k) = sample_rate / k ; num_channels =k; },
• so `channelizer(channelizer_rank) is the one and only channelizer created in CubicSDR as in now.

For instance for a 2.4MHz RTL-SDR, that leads to the following possible channelizers: channelizer(4) is the 500KHz one like today, the next ones are channelizer(3), channelizer(2). For channelizer(1) it is equivalent to having none, so is never used but runSingleCH() is used instead.

For a 10MHz Airspy/SDRPlay, we have the following possible channelizers:

• channelizer(20) = {channelizer_bw(20) = 500KHz ; num_channels =20; }
• channelizer(19) = {channelizer_bw(19) = 526KHz ; num_channels =19; }
• channelizer(18) = {channelizer_bw(18) = 555KHz ; num_channels =18; }
• channelizer(17) = {channelizer_bw(17) = 588KHz ; num_channels =17; }
• channelizer(16) = {channelizer_bw(16) = 625KHz ; num_channels =16; }
• channelizer(15) = {channelizer_bw(15) = 666KHz ; num_channels =15; }
• ...
• channelizer(5) = {channelizer_bw(5) = 2000KHz ; num_channels =5; }
• channelizer(4) = {channelizer_bw(4) = 2500KHz ; num_channels =4; }
• channelizer(3) = {channelizer_bw(3) = 3333KHz ; num_channels =3; }
• channelizer(2) = {channelizer_bw(2) = 5000KHz ; num_channels =2; }

The idea is to provide not only increasing BW, i.e less edges, but ones that assure the least chance to have aligned channel edges for some unlucky frequencies. Now we try to define a demod_fit criteria of how good a demod is fitted to its affected channelizer(k). The ideal fitting is of course when a demod freq is the same as the channel center freq it belongs to, i.e when both are centered on each other. One of the worst situations is when the demod edge is close to the channel, or worse, overlaps it.

So I've come to the demod_fit = (freq_distance + 0.5 * demod_bw) / (0.5 * channelizer_bw(i)) where freq_distance = abs(demod_center_freq - channel_center_freq).

That leads that demod_fit is not only minimal for a centered channel on the demod center (freq_distance = 0), but also measure the channel 'relative' occupation, ex. demod_fit > 1 means the demod overlaps the frontiers of its channel.

So the algorithm is as follows to assign a demod to a channelizer:

1. Start at k = max_channelizer_rank = int(sample_rate / CHANNELIZER_RATE_MAX).
2. Create (if not exist) channelizer(k)
3. Search to the best fitted frequency for demod_freq among the channelizer(k) channels, as the current algorithm does.
4. Computes the demod_fit(k): 4.1. If demod_fit(k) <= max_demod_fit (ex 0.7) then channelizer(k) is fine for this demod, exit. else 4.2. demod_fit(k) > max_demod_fit the demod is too cramped in the channnelizer(k). So, k := k - 1, return to 2 if k > 1, else choose the channelizer(k) for which demod_fit(k) was minimal.
5. Cleanup : if after deleting a demod its channelizer(j) is empty of other demods, delete channelizer(j).

Pros:

• Each demod find the smallest bandwidth channelizer it actually fits in, minimizing the wrapped- up ghosts.
• May be applied as-is on CubicSDR without considering additional 'performance' options.

Cons :

• Demods have different BW depdending on the channelizer they are assigned to.
• More CPU intensive that single channelizer solution, obviously.

[cjcliffe]

@vsonnier I like the idea; but the channelizer is rather expensive computationally and trying to run them in parallel on the main input is going to pile up quickly..

I'd like to start with just a simple frequency shift of the main input that will adapt to the current set of dmods in view. I think a lot of it can be resolved by just frequency shifting so the boundaries are pushed into areas unoccupied by (visible) demods -- which can also be done dynamically without interrupting streams or adding a ton of CPU cycles.

In any case I would like to avoid doing anything that interrupts demod streams which could interfere with tools/recorders attached to the outputs.

--

The next solution if it's a "high cpu mode" would be to use firpbch2 -- which is a double-bandwidth channelizer where the channels fully overlap each other and there are no well-defined gaps as you can use whichever channel the demod is "more" contained in.

Unfortunately it doesn't perform nearly as well as the firpbch but likely better than 2+ firbpch and could easily be enabled as a third path for the post thread, since it's a near drop-in replacement (and I've already had it working before) that just requires some channel variable adjustments.

--

The final (and more complicated) solution I had planned for this is to re-combine channels as needed (the channelizer works in reverse too ;) which allows you to make on-demand high-bandwidth channels by combining a set of channels from the main channelizer back into a single stream -- I think this would also be more efficient than multiple channelizers on the main input.

I'll probably keep this issue to try next once I've got some audio recording going -- unfortunately dealing with a dying Macbook and setting up a new PC laptop with the build environment here at the moment.

[vsonnier]

@cjcliffe Thanks for the explanation ! I'm still staring at firpbch2 examples and API, and still do not understand it :) Anyway, I'm looking forward to squash your future audio recording bugs, as usual. Joking aside I am happy to contribute that way, I just wish I could understand DSP better. See you !