Input source type of 'rtlsdr' is not supported

[srs4511351]

Running a Raspberry Pi 4 Raspberry Pi OS Debian GNU/Linux 12 (bookworm)

On the latest version of pyspectrum, the device 'rtlsdr' is not supported. rtlsdr does work with soapy.

I was able to use it on older installations.

One difference is that on bookworm, they no longer allow (or strongly discourage) installation with pip. I use this procedure:

Download and extract the source archive from https://pypi.org/project/pyrtlsdr/#files

cd ~/pyrtlsdr-0.3.0
sudo python3 setup.py install

#Test installation:
python3
import rtlsdr
# There should be no error
^D to exit.

It does import rtlsdr with no error.

$ python ./src/SpectrumAnalyser.py -i?

Available sources: ['file', 'null', 'rtltcp', 'soapy', 'socket']
file:Filename   - Filename, binary or wave, e.g. file:./xyz.cf123.4.cplx.200000.16tbe
null:xxx    - xxx ignored parameter
rtltcp:IP:port - The Ip or resolvable name and port of an rtltcp server, e.g. rtltcp:192.168.2.1:12345
soapy:Name  - Name is soapy driver, e.g. soapy:?, soapy:sdrplay
socket:IP:port  - The Ip or resolvable name and port of a server, e.g. socket:192.168.2.1:12345

[naj1024]

Mmmm... Is there anything in the SpectrumAnalyser.log file under the logs directory? If nothing there then probably the interface has changed, bit-rot! Do you use any other rtldr software on the pi, maybe it is the low level driver that is missing or not enabled.

I will endevour to take a look soon, have to dig out a pi. Glad soapy works for you, never had much luck with soapy but i tend to develop under windows.

The default git branch was changed recently from main2 to flask - so could be i introduced an error somewhere. You could try the main2 branch. Having said that :

I would check the log file first.
Then try another app that uses rtlsdr, something not using python so we check the low level driver out.
Try main2 branch.

[srs4511351]

SpectrumAnalyser.log

2023-12-15 05:50:04 UTC,ERROR:root:DataSource_funcube:funcube source has an issue: No module named 'sounddevice'
2023-12-15 05:50:04 UTC,WARNING:root:DataSource_funcube:Warning: funcube source has no ability to control device, No module named 'hid'
2023-12-15 05:50:04 UTC,ERROR:spectrum_logger:DataSource_rtlsdr:rtlsdr library search path was: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
2023-12-15 05:50:04 UTC,ERROR:spectrum_logger:DataSource_rtlsdr:rtlsdr source has an issue: Can't find library rtl
2023-12-15 05:50:04 UTC,ERROR:root:DataSource_audio:audio source has an issue: No module named 'sounddevice'
2023-12-15 05:50:04 UTC,ERROR:spectrum_logger:DataSource_pluto:pluto source has an issue: No module named 'adi'
2023-12-15 05:50:54 UTC,ERROR:webUI.WebSocketServer:WebSocketServer:WebSocket socket Tx exception for 127.0.0.1, received 1001 (going away); then sent 1001 (going away)
2023-12-15 05:51:01 UTC,ERROR:spectrum_logger:SpectrumAnalyser:SpectrumAnalyser exit

I have these packages installed

rtl-sdr/stable,now 0.6.0-4 arm64 [installed]
librtlsdr-dev/stable,now 0.6.0-4 arm64 [installed]
librtlsdr0/stable,now 0.6.0-4 arm64 [installed]

The rtlsdr works in qradiolink and gqrx with the device string of rtl=0

I found that pyspectrum works on rtlsdr with the main2 branch and fails with the flask branch.

SpectrumAnalyser.log with main2

2023-12-17 05:29:09 UTC,WARNING:root:DataSource_funcube:Warning: funcube source has no ability to control device, No module named 'hid'
2023-12-17 05:29:35 UTC,ERROR:webUI.WebSocketServer:WebSocketServer:WebSocket socket Tx exception for 127.0.0.1, received 1001 (going away); then sent 1001 (going away)
2023-12-17 05:29:35 UTC,ERROR:webUI.WebSocketServer:WebSocketServer:WebSocket server process exited
2023-12-17 05:29:35 UTC,ERROR:spectrum_logger:SpectrumAnalyser:SpectrumAnalyser exit

[naj1024]

Think i have found the issue, on the flask version i tried to check for librtl instead of librtlsdr. On main2 i was loading rtlsdr differently. My bad, i have checked in the change so a git pull should now work for the flask version. Found issues with pluto support and audio as well, seems to always be problems with libraries the python modules depend on. Don't seem to be able to catch all exceptions.

[srs4511351]

rtlsdr work on the flask branch now. There are some interesting things that I noticed. When I switch to use rtlsdr thru soapy, I get data overruns evidenced by a series of O characters in the terminal. They do not appear with the direct rtlsdr device. I need soapy because my main SDR is SDRplay.

Also, it is very difficult to change the device. It usually switches back to the previous device and only works after several tries. This may be caused by the CPU running at 92% on the Raspberry Pi 4 overclocked to 1800 MHz.

[naj1024]

Thanks for the feedback. As you point out it is probably cpu load making things difficult when trying to switch sources. You could try using 'stop' on the analysis tab before switching sources as this starts dropping samples and should reduce cpu load. Alternatively select the null source before the next one. As this is predominately python it's not fast, or well written it definatley has issues :|

[srs4511351]

With Analysis stopped, and trying to change the Source device to Soapy, the pull-down keeps changing to null when I press Attach. After several tries, to works. Still getting overruns.

[naj1024]

Could you run with -vvv (three v's to enable debug) and copy the SpectrumAnalyser.log file here.

[srs4511351]

I think the problem was due to misspelling "sdrplay". This time it didn't like sdrplay being in the text field when rtlsdr was selected. It is probably working correctly. I wonder if something can be done about the data overruns?

SpectrumAnalyser.log

[naj1024]

I have headless pi 3 to test with, so should be even slower than your pi 4 - however:

At 1MSps and 2048 FFT we get 2048usec for each sample buffer. So we need to complete everything in under 2048usec (488 spectrums per second) rtlsdr read of 1000usec and processes time of around 1000usec and free of about 22% after all other operations

 from: 2023-12-29 17:04:17 UTC,DEBUG:spectrum_logger:SpectrumAnalyser:SPS:1000000, FFT:2048 2048usec, read:1074usec, process:981usec, analysis:209usec, report:73usec, snap:98usec, ui:241usec, total:1602usec, free:22%, pc:23.0, fps:20, mfps:11.9, 

Your similar config on the pi 4 gave: rtlsdr read of 1500usec and processes time of around 1500usec and free of around -24%

from: 2023-12-24 04:12:34 UTC,DEBUG:spectrum_logger:SpectrumAnalyser:SPS:1000000, FFT:2048 2048usec, read:1940usec, process:1348usec, analysis:453usec, report:10usec, snap:212usec, ui:492usec, total:2515usec, free:-23%, pc:23.0, fps:20, mfps:6.0, 

And for soapy sdrplay rtlsdr read of 1500usec and processes time of around 2100usec and free of around -70%

from: 2023-12-24 04:13:04 UTC,DEBUG:spectrum_logger:SpectrumAnalyser:SPS:1000000, FFT:2048 2048usec, read:1534usec, process:2056usec, analysis:623usec, report:17usec, snap:172usec, ui:539usec, total:3407usec, free:-66%, pc:23.0, fps:20, mfps:5.0, 

For some reason the reading of samples is a lot slower and processing as well is slower. Unless we get a free of >0% then we are not running fast enough. Sometimes a larger fft will work better, say 4k, especially if pyfftw is installed.

• Maybe the overclocking is causing thermal throttle to happen?
• I think you are running the web browser on the Pi as well (connection from 127.0.0.1) maybe we are thrashing swap?

I made a few changes and merged them into flask branch. Nothing that has impacted the problem, just code clean up really.

This is where its obvious that python is not the language to do this is, but it is nice and portable between windows and linux.

I was hoped that pyfftw could be used to speed things up, but so far i have found that enabing multiple threads makes things a lot worse. Probably becasue i don't have aligned memory (yet).

My PI-3

$ cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" NAME="Debian GNU/Linux" VERSION_ID="12" VERSION="12 (bookworm)" VERSION_CODENAME=bookworm ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"

$ cat /proc/cpuinfo | grep Model Model : Raspberry Pi 3 Model B Rev 1.2

$ free -m total used free shared buff/cache available Mem: 908 466 81 4 425 441 Swap: 99 73 26

$ watch -n 1 vcgencmd measure_clock arm frequency(48)=1200000000

[srs4511351]

Switch to SoapySDR and try again.

I do not get overruns on the native RTL-SDR device.

Is there a way to run it without the web server?

My RPI 4 has extra heat sinks and a fan. It doesn’t get hot enough to throttle. It acts the same on the OS that isn’t overclocked.

Is there a specific test that you want me to perform?

[naj1024]

Can't think of another test, essentialy it is not reading fast enough. I believe it will be dropping samples in rtlsdr as well, it is just not obvious.

The web server part is in another processes, not thread, so shouldn't impact anything. If you don't connect to it it wont be doing anything, although i still send spectrums to the web process. It would take a re-write to make everything faster, i.e. swap out most of the python for C/C++.

If run you Pi headless and connect to the web server from another machine it may work better as other processes won't be running, assuming you have a desktop on the Pi currently and its compatible with your use.

I'm going to close this issue, as the original issue is solved, just highlights another issue as well.

I have added a -Dn option, where n is drop 1 in n buffers read from the device, e.g. -D2 will drop every other buffer read. This may produce spectral artifacts e.g. a tone would have phase changes every n buffers, which would widen the spectral line. Obviously you will loose short events randomly if they are in a dropped buffer. This saves all the fft and follow on processing time which should cut down on the cpu load.

Thanks for your patience

[srs4511351]

None of this addresses why soapy has more significant dropouts than the native RTL-SDR api. If you would try it, you might see this. I will play with the new switch. It may be useful.

[srs4511351]

I tried the -D2 switch and could not see much difference. CPU load is 85%-90%, so it is overloaded.

Is there a limit for the number in the -D switch?

[naj1024]

-D2 will drop 50% of the buffers. That should of had the most impact. There is no limit but -D10 would only drop 1 in 10 so less impact. I can't currently test soapy as my pi-3 is also my pihole/dns and it really struggles with pyspectrum - such that i loose internet, very unpopular around here. Just ordered a pi-5 so should be able to investigate further soon, sorry but didn't see the point of getting as pi-4. Try increasing the fft size to 4k/8k to see if that helps, shorter ffts will make things worse as overheads are a bigger percentage at short fft sizes.

[naj1024]

Built raspberry pi (5) and updated sdrplay source. Didn't see any changes in use of sdrplay once i had installed the sdrplay properly. I noticed that soapy blocks up the sample data which was causing my debug to show that the reading of samples was taking too long, changed that so it reports more correctly. Modified how i read soapy samples as well to reflect this blocking up.

On the pi5 i start to see dropped samples at 5Msps (2048 fft). Using SoapySDRUtil the raw read was not dropping samples even at 8Msps SoapySDRUtil --rate=8e6 --direction=RX --args="driver=sdrplay"

If the raw read on the pi-4 tops out differently then that may be the problem.

I updated the readme.md to give a complete record of how i installed on the Pi.

I'm still running headless as i don't have a micro-hdmi cable, should have one soon.

If you want to try this release: git pull git checkout soapy_features

[srs4511351]

I did git pull git checkout soapy_features

It acts the same for me. My Pi4 CPU load is near 100%, however, closing the browser drops CPU load to 50% and I still get data overruns. Changing fft to 4k, or -D2 decimation don't make a noticeable difference.

Running it on a Pi 5 is not a comparison for the Pi4 or earlier, which most people have. I am running the SDRplay API 3.07. #.12 if very recent and requires rebuilding the SDR system. Changing the API will leave the system not working. SoapySDR packages are installed with GNU Radio. I installed libsoapysdr-dev soapysdr-tools packages https://github.com/SDRplay/SoapySDRPlay.git https://github.com/SDRplay/SoapySDRPlay.git is redirected to https://github.com/pothosware/SoapySDRPlay3.git, which is what we both installed.

My installation is here: https://www.radiosrs.net/

pyspectrum is on this page https://www.radiosrs.net/installing_utility_software.html

Drivers, Soapy, etc. are here: https://www.radiosrs.net/installing_drivers_sdrplay_and_HackRF_Bookworm_RPI_OS-64.html

I can run PLSDR, which is all Python scripts using Gnu Radio. It works well, but does not work with Gnu Radio > 3.8 My systems work well with that. With SdrGlut, I can receive at 10 Mbps with only occasional audio dropouts.

What will I see if is is dropping samples with these tests? SoapySDRUtil --rate=8e6 --direction=RX --args="driver=sdrplay" Begin RX rate test at 8 Msps ... 7.5673 Msps 30.2692 MBps 7.78182 Msps 31.1273 MBps

SoapySDRUtil --rate=10e6 --direction=RX --args="driver=sdrplay" Begin RX rate test at 10 Msps ... 9.42812 Msps 37.7125 MBps 9.70989 Msps 38.8395 MBps 9.80971 Msps 39.2388 MBps

What do you mean by "installed the sdrplay properly?"

[naj1024]

Thanks for the quick response.

I agree RPI5 is not a replacement for a RPi4, as i had to buy a Pi and i wasn't going to go with a RPi4 when for $5 more i could get the faster one. I may be able to convince someone to lend me a RPi4.

Sorry, "install sdrplay properly" was because i had real problems getting it installed - hence building from source. Until i did that the sdrplay (an old RSP1) i have was being identified as a miri device and was uncontrollabe, fixed at 8Msps and lots of overruns. Once the driver was available i could at least talk to it and configure it correctly. I didn't install lots of other sdr software as i wanted to keep it clean and know exactly the dependancies.

Running SoapySDRUtil was to try and identify if it was a lower level problem than pyspectrum, ie could the basic read and throw away of the test utility keep up. We can both run at 8Msps without dropping with this utility, (RSP1 only goes to 8Msps) so not a low level problem. If you are dropping samples it will print 'O''s to the prompt and say dropped samples at the end.

So it is the python pyspectrum code that is the problem.

I agree that messing with your current install is not an option, too many interdependencies. I had hopped that the rewrite of the soapy input module would help. I didn't see any difference on thr RPi5 but it highlighted some edge cases not handled in the code and the fact that the soapy interface blocks things up was news to me.

There are a few more things i can try to identify the problem. Like start to cut out particular parts of the code, run a desktop at the same time (waiting on the micro hdmi cable). It is interesting that most reviews i've seen on the RPi5 say it is around 4 times faster - and i start to see problems at 5Msps with the sdrplay. So maybe it is just the way i have written the python. Most of the time seems to be spent on reading the samples on the Rpi5 so i will write a simple read and bin example to get some familiarity with that.

Could you give me a SpectrumAnalyser.log with this git branch using -vvv?

[srs4511351]

Thanks for your comments. It appears that I may have things installed correctly. Congratulations on getting a RPI 5. I am waiting until my finances settle down...

I changed to a Raspi OS system with the SDRplay API 3.12. Please see attached. SpectrumAnalyser.log

[naj1024]

Thanks for the log. A few things for me to look at. Post holiday finances can be a challenge.

[naj1024]

I managed to borrow a Pi4 and run the code. Yes it drops samples, as far as i can tell the Pi4 is just not up to it, especially if it is running a local desktop with chrome browser. I created a docs directory with some timings between the pi4 and pi5. Its not memory its just cpu time, copying things around and doing the computations.

Only way out of this would to section the project into a C (or another language) and python split with most of the work done in C. As this was a test python project i'm not going to go that far. (for the pluto one option would be to use https://maia-sdr.org/installation/ )

I still think the code has some uses, i find it handy for timing things in the spectrum.

I noted while debugging on the Pi4 were that you can't gauruntee that any source device is not dropping samples. Soapy can do well in detecting this but some devices will drop samples without any warnings. I use the pluto device and that definately just drops samples with no warnings - only way out on that one would be to rewrite the fpga part to give counters of dma buffer overflows. Soapy rtlsdr does better than the standard rtlsdr driver which never detects that its not keeping up.

I at least got the sdrplay soapy to display the overflows (dropped buffers) and i added in loop % which when everything is running correctly should be close to 100%. I wouldn't go over 1Msps on a Pi4 withou/t checking out the sample_rates_tests.txt file in the docs directory.

While testing i did install something which completely destroyed performance on the Pi4, i was searching system paths as well as the virtual environment. I think i installed something that was slow compared to what i had in the virtual environment. Took a reinstall to get it back working again. Python seems to be tricky on its installations, so virtual environments are the way to go.

Thankyou for your feedback and from this i have definately improved the code and my understanding of python and the problem space.

The latest code is under the flask branch.

I will close this issue as i think i have reached the end of what i can do. cheers

[srs4511351]

Could you help me with the tricky Python installations? Another developer just left me to figure it out, but there is no solution that case. I have been having trouble installing Python modules in a virtual environment. If I use the recommended pipx, which uses a virtual environment, to install a Python module, it is not available for import without special handling. Installing with sudo python3 setup.py install works the best, but now, they want to deprecate setup.py! That only leaves me with using a package in the distro or virtual environment which is far from optimal.

Thanks for checking it out for the Pi. It would be nice to have a spectrum analyzer, but I haven't been able to find anything but pyspectrum. Do you know of any? It's still useful like it is for just viewing spectrum.

[naj1024]

I revised the readme for the project and included everything i did to install on a Pi. Part of that is to do the virtual environment. Does that cover where you are having difficulty?

Virtual environments in python are, i believe, designed to make it easier to have multiple competing python environments as each is separate. Installing it all as root breaks things as one will clash with a previous one.

Other spectrum analysers softeware is available that should work on the pi. Most are written in C/C++ so dont have such performance constraints as pyspectrum in python. Try GQRX, there are also prebuilt pi images with ham radio tools preinstalled, googled and found this from 2021 https://maker.pro/raspberry-pi/projects/diy-sdr-dsp-radio-with-raspberry-pi-and-rtlsdr-dongle . There are others.

Python vitual environment: sudo apt install pipenv pipenv shell

• Ctrl-D when you have a prompt exits the shell and this will have setup the environment in ~/.local/share/virtualenvs/pi-xxxxxx where xxxxxx is unique this is where all the python packages will be installed for the environment.

then to install python packages into the environment cd pipenv shell pip install numpy (as an example)

python3 import numpy

and it should be there.

[srs4511351]

I forgot to mention that Debian Bookworm insists that you do not use pip install. The only way around that is to use pip install --break-system-packages It looks to me like Python or the OS managers are trying to KILL Python by making it nearly impossible to use. I do not install more than one version of Python packages, as I have not needed to yet.

Is it permissible to use the forbidden pip install in a pipenv shell as your procedure call for?

I will try this out later.

[naj1024]

I revised the readme for the project and included everything i did to install on a Pi. Part of that is to do the virtual environment. Does that cover where you are having difficulty?

Virtual environments in python are, i believe, designed to make it easier to have multiple competing python environments as each is separate. Installing it all as root breaks things as one will clash with a previous one.

Other spectrum analysers softeware is available that should work on the pi. Most are written in C/C++ so dont have such performance constraints as pyspectrum in python. Try GQRX, there are also prebuilt pi images with ham radio tools preinstalled, googled and found this from 2021 https://maker.pro/raspberry-pi/projects/diy-sdr-dsp-radio-with-raspberry-pi-and-rtlsdr-dongle . There are others.

Python vitual environment: sudo apt install pipenv pipenv shell

• Ctrl-D when you have a prompt exits the shell and this will have setup the environment in ~/.local/share/virtualenvs/pi-xxxxxx where xxxxxx is unique this is where all the python packages will be installed for the environment.

then to install python packages into the environment cd pipenv shell pip install numpy (as an example)

python3 import numpy

and it should be there.

[naj1024]

Commenting on use of pipenv and pip install. As the python packages within a virtual environment (pipenv shell) are completely separate from the OS python install then this is the safest and best way to use python. I can't understand Debian not recommending virtual environments, either they have something better or there is some mis communication.

At the end of the day, its your computer/OS and you can do what you like. It may make things interesting if it diverges a lot form what debian expect you to do. But i can't see it making much difference for now.

[srs4511351]

Thank you for saving my installation and your tests on the RPI.

Your virtual environment procedure seems to be working.

As for the pain and counterintuitive nature of deprecating traditional ways of installing Python modules, I think it is mainly for developers. On my system, there is presently only one installation of each module I use, just like the other packages, python or otherwise, installed by apt. I don't need multiple installations of the same thing.

Debian does recommend using a virtual environment, but they recommend using pipx to manage virtual environments.

I write installation procedures and put them on my web page for the public to use. This is a good example of why I do it.

With Linux, I have seldom or never been able to install a compiled application without some sort of trouble. I find out what the provided instructions left out and give it on my web page.

Having searched the web for a good solution for pip install, I didn't find any. The people on my web page need the same help, but I do the hard work for them.

Thanks again.

[naj1024]

investigated and more of a resource issue than a bug