robotastic / trunk-recorder

Records calls from a Trunked Radio System (P25 & SmartNet)
GNU General Public License v3.0
834 stars 191 forks source link

Polyphase Channelizer #76

Open ddb opened 7 years ago

ddb commented 7 years ago

In the interest of reducing CPU load and increasing the number of possible recorders, would it help matters to use a polyphase channelizer to split out the signal for each recorder?

I am admittedly a relative newbie hear, so I could be easily talking about matters of which I know very little, but when I read in the README about how each record gets the entire bandwidth and has to pull it's desired piece out of it, it sounded to me like a polyphase channelizer would be the ideal solution to this problem, allowing you to filter, decimate, and translate to baseband in one step per channel, and then not having it feed the entire bandwidth to each channel.

https://casper.berkeley.edu/wiki/The_Polyphase_Filter_Bank_Technique

ddb commented 7 years ago

Doing some research, I see that CubicSDR is already using this technique to make it easier to handle multiple demodulators. It seems that Liquid-DSP provides polyphase filter bank functions.

robotastic commented 7 years ago

I looked into using PFBs but couldn't get it to work. I will check and see how CubicSDR uses it. Maybe there is some code I can borrow from it. I am not sure if PFBs are able to channelize large swatches of bandwidth, like 8MHz.

If you are able to tinker with it too, that would be awesome! http://www.trondeau.com/examples/2014/1/23/pfb-channelizers-and-synthesizers.html http://gnuradio.org/doc/doxygen/page_pfb.html

On Sun, Dec 4, 2016 at 6:16 PM, David Brown notifications@github.com wrote:

Doing some research, I see that CubicSDR is already using this technique to make it easier to handle multiple demodulators. It seems that Liquid-DSP provides polyphase filter bank functions.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/robotastic/trunk-recorder/issues/76#issuecomment-264739791, or mute the thread https://github.com/notifications/unsubscribe-auth/AAG53OuiwRJpLmtbwGYf4XYTyak-5tfJks5rE0nJgaJpZM4LDEVm .

ddb commented 7 years ago

I do plan to tinker.

It looks like the way Liquid does the feature, it calculates the signals for every channel, and it won't be much of a savings if we do that over the entire bandwidth. I can see adding a version that takes a list of desired channels as part of the parameters, and then only calculating the results for those channels, thereby saving CPU we would have otherwise spent calculating unneeded data.

I think I can see how to do it in the existing liquid code, but I admit to not really knowing if it will mess up the math in some inscrutable way and I find myself out of my depths.

And first I really have to understand how to use it to begin with, so there's that.

boxlor commented 7 years ago

PFB channelizers are definitely a very CPU-efficient method of capturing many trunk channels simultaneously! I have used them in gnuradio (the hierarchical PFB channelizer block) on a couple of different P25 systems and RTL dongles. Here's an example flowgraph that does the channelization and demod for 20 frequencies simultaneously using three RTL dongles to cover enough bandwidth for an entire trunk system. I bring the output into dsd via UDP packets and use a command like:

nc -l -u localhost <port> | dsd -f1 -i - -w outfile.wav

Each of the 20 PFB outputs gets its own UDP port and dsd instance.

By decoding all of the trunk channels at once there's no need to bother with the control channel. (I modify dsd to split the recordings into separate files and insert them into the database.)

example.grc.txt

example grc

edit: updated to match corrected gist below.

robotastic commented 7 years ago

Whoa!!!! This is amazingly awesome. I will go try playing around with this over the weekend. I have been using FFT filters instead of FIR for channelization. I will go give this a try.

Would the CPU usage be the same for 3 RTLs at 2MSPS as 1 HackRF running at 6 MSPS?

On Jan 13, 2017, at 10:28 PM, boxlor notifications@github.com wrote:

PFB channelizers are definitely a very CPU-efficient method of capturing many trunk channels simultaneously! I have used them in gnuradio (the hierarchical PFB channelizer block) on a couple of different P25 systems and RTL dongles. Here's an example flowgraph that does the channelization and demod for 20 frequencies simultaneously using three RTL dongles to cover enough bandwidth for an entire trunk system. I bring the output into dsd via UDP packets and use a command like:

nc -l -u localhost | dsd -f1 -i - -w outfile.wav

Each of the 20 PFB outputs gets its own UDP port and dsd instance.

By decoding all of the trunk channels at once there's no need to bother with the control channel. (I modify dsd to split the recordings into separate files and insert them into the database.)

example.grc.txt https://github.com/robotastic/trunk-recorder/files/705722/example.grc.txt https://cloud.githubusercontent.com/assets/23716476/21952001/cfdb072a-d9c5-11e6-8e95-036dde010809.png — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/robotastic/trunk-recorder/issues/76#issuecomment-272598330, or mute the thread https://github.com/notifications/unsubscribe-auth/AAG53E1S6LAXAO1BfDtxnQlBAVKwVkdtks5rSEDOgaJpZM4LDEVm.

boxlor commented 7 years ago

In case it's helpful, here's the script I used to determine which PFB outputs to plug into: https://gist.github.com/boxlor/02deb422096e98a63d284499c727f53c

Let me know how it goes!

edit: updated gist to correct error when number of channels even.

robotastic commented 7 years ago

That is really helpful! I think I screwed the channel selections up last time I tried. I will keep you posted on how it goes. Thanks again!

On Jan 13, 2017, at 11:26 PM, boxlor notifications@github.com wrote:

In case it's helpful, here's the script I used to determine which PFB outputs to plug into: https://gist.github.com/boxlor/fbdb5b405b2f7594b0ab8af1f9ee24e2 https://gist.github.com/boxlor/fbdb5b405b2f7594b0ab8af1f9ee24e2 Let me know how it goes!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/robotastic/trunk-recorder/issues/76#issuecomment-272600984, or mute the thread https://github.com/notifications/unsubscribe-auth/AAG53CRmORb4P8_jFaLAb0E_faXvhhWAks5rSE5sgaJpZM4LDEVm.

imagesafari commented 7 years ago

There's got to be a way to use a GPU instead of CPU for some of this stuff too, no?

boxlor commented 7 years ago

Wow, cool idea. Typically it's done in an FPGA but you're right, PFBs have been done in GPUs as well. I wonder if there are any examples to start from.

http://www.haystack.mit.edu/edu/reu/2011/files/mccurry.pdf http://asp.eurasipjournals.springeropen.com/articles/10.1186/1687-6180-2014-141 http://gnuradio.org/redmine/attachments/download/458/130606-VT-GNUradio_HackerFest.pdf

boxlor commented 7 years ago

Also a good overview starting in section 4.1.3 of: Distributed wideband software-defined radio receiver for heterogeneous systems

vabiro commented 6 years ago

Hi guys, I am curious if this made it into the project.

I am working with a large system (http://bit.ly/2zeEtJU), with 4 sites and 52 channels. While activity isn't large (i.e. a happy day for a Motorola salesman), the multiple sites do seem to make effectively listening to more than a few TGIDs, or more than one site, a problem for CPU usage.

Being able to offload some of the burden to the (cheaply upgradable) GPU would be a welcome option, as well as the idea getting all the stuff, and discarding the unwanted, may work more efficiently than what is happening right now.

Thanks again Luke et al for all your work. I hope to contribute soon.

robotastic commented 6 years ago

Unfortunately not... if anyone is able to package it up in a nice and generalizable way, I would be happy to add it in. Using a PFB to channelize multiple MHz of spectrum doesn't seem to be any more efficient using a CPU. It would take a bit of study to look at the trade-offs. The SDR they are using in the MIT study is super expensive. It would be cool to get it working on a LimeSDR's FPGA.