pothosware / SoapySDRPlay3

Soapy SDR plugin for SDRPlay APIv3
https://github.com/pothosware/SoapySDRPlay3/wiki
MIT License
98 stars 15 forks source link

Overflow with RSP1A approx. 4x per hour --> Single short audio pieces are missing in the stream (fraction of a second) #36

Closed thowe-switzerland closed 3 years ago

thowe-switzerland commented 3 years ago

RTLSDR-Airband (latest unstable) SoapySDR 0.8.0 SoapySDRPlay 0.3.0 SDRplay_RSP_API-ARM32-3.07.2

I am currently doing tests for a new LiveATC streaming setup. I am using two SDRs in parallel with RTLSDR airband via SoapySDR: an RSP1A and an Airspy mini.

With the official version of RTLSDR-Airband it always stopped after about 5 to 15 minutes with the following message: OSoapySDR device 'driver=sdrplay,biasT_ctrl=false': readStream failed: OVERFLOW, disabling

The author of RTLSDR-Airband then referred me to the current unstable branch, which does not stop in such a case. And indeed: RTLSDR-Airband now runs, even when overflows occur at RSP1A.

Since the Airspy mini is running in parallel, I have two more or less identical MP3 files, which I record. However, I notice the following: It is always recorded for an hour in parallel into an MP3 file. While at the beginning of an hour both files are absolutely at the same time, the RSP1A file is a few seconds ahead (about 3 seconds) until the end of the hour. I then noticed that per hour maybe 4 to 5 times short moments (fraction of a second) are missing in the stream of the RSP1A. This manifests itself as a few syllables being swallowed each time. This then seems to add up over an hour.

I assume that the swallowed short time segments always occur when an overflow occurs. Since the Airspy mini does not have this problem: Is there any way to tell what is causing these sporadic overflows and what I can do about it?

fventuri commented 3 years ago

@thowe-switzerland - very interesting application using RTLSDR airband.

Just to make sure I know which code you are running for the SoapySDRPlay driver - by "current unstable branch" do you refer to the master branch of this repository or to a different branch?

Also could you post a few lines of the current output (i.e. with the code from SoapySDRPlay 0.3.0) when running the RSP1A that show the exact OVERFLOW warnings?

Thanks, Franco

thowe-switzerland commented 3 years ago

Thank you Franco!

I took the version number "0.3.0" from the document Changelog.txt, because I found no other version. I built SoapySDRPlay according to the instructions in the wiki. The version checked out with git is from my point of view the current head of the repo on the master branch:

pi@atc-stream-rpi:~/SoapySDRPlay $ git log
commit e6fdb719b611b1dfb7f26c56a4df1e241bd10129 (HEAD -> master, origin/master, origin/HEAD)

For SoapySDRPlay, I checked out and built exactly as instructed on the wiki - the master branch. I had to choose the unstable branch only for RTLSDR-Airband, because only this version continues to run after an overflow in SoapySDR.

Currently I don't have much information or log entries. What I have you can find in the ticket at RTLSDR-Airband: [BUG] Stops with OSoapySDR device 'driver=sdrplay': readStream failed: OVERFLOW, disabling

If you want me to build a special version, adjust the log level or anything else to help with the analysis, I would appreciate a hint on how to proceed. I will be happy to do that then.

Many greetings Thomas

fventuri commented 3 years ago

@thowe-switzerland - Thomas, thanks for the detailed info.

To troubleshoot this problem, I created a new branch called debug_overflows (https://github.com/pothosware/SoapySDRPlay3/tree/debug_overflows) - as you can see in the commit (https://github.com/pothosware/SoapySDRPlay3/commit/f263cff1ebc01c19b001455ba80fdc04572c8e24), the only difference between this branch and master is a couple of log warnings that you should see on the terminal whenever on overflow occurs inside the SoapySDRPlay3 driver.

Please build this branch, and run the RTLSDR-Airband as usual for an hour or so with the same settings (sample rate, etc) you used before, and let me know if you see those warning messages.

Also I'd like you to run (again with the code from this branch debug_overflows), the program SoapySDRUtil to test streaming without any other processing (that could add overhead), like this:

SoapySDRUtil --args='driver=sdrplay,biasT_ctrl=false' --rate=8e6 --direction=RX

Please let this one too run for an hour or so, and let me know if it reports any overflow warning.

As per the version number, you have a valid point - I should update the file Changelog.txt, and assign a git tag for the current master branch, so there is no ambiguity.

Franco

thowe-switzerland commented 3 years ago

Wow! Thanks Franco for the great support with my problem.

I was able to check out, build and install the branch without any problems.

Here is how I proceeded:

  1. make uninstall in the build directory of the old installation
  2. checked with SoapySDRUtil --info that SoapySDRPlay module is no longer found
  3. rm -rf SoapySDRPlay
  4. build & install according to instructions with an additional checkout debug_overflows.

Result after installation of the special built module - the commit hash of the debug_overflows module is active:

pi@atc-stream-rpi:~ $ SoapySDRUtil --info
######################################################
##     Soapy SDR -- the SDR abstraction library     ##
######################################################

Lib Version: v0.8.0-g3fa5ef74
API Version: v0.8.0
ABI Version: v0.8
Install root: /usr/local
Search path:  /usr/local/lib/SoapySDR/modules0.8
Module found: /usr/local/lib/SoapySDR/modules0.8/libairspySupport.so  (0.1.2-10d697b)
Module found: /usr/local/lib/SoapySDR/modules0.8/libsdrPlaySupport.so (0.3.0-f263cff)
Available factories... airspy, sdrplay
Available converters...
 -  CF32 -> [CF32, CS16, CS8, CU16, CU8]
 -  CS16 -> [CF32, CS16, CS8, CU16, CU8]
 -  CS32 -> [CS32]
 -   CS8 -> [CF32, CS16, CS8, CU16, CU8]
 -  CU16 -> [CF32, CS16, CS8]
 -   CU8 -> [CF32, CS16, CS8]
 -   F32 -> [F32, S16, S8, U16, U8]
 -   S16 -> [F32, S16, S8, U16, U8]
 -   S32 -> [S32]
 -    S8 -> [F32, S16, S8, U16, U8]
 -   U16 -> [F32, S16, S8]
 -    U8 -> [F32, S16, S8]

Surprisingly, I did not find any hints in the output. I started both on the shell with rtl_airband -Fe and as a service. Neither on the console nor in /var/log/messages nor in the service log I found any hint. The audio files show the effect though. I.e. the RSP1A was missing samples during this time. I have heard a few places where single syllables are missing compared to the recording with the Airspy mini.

The same is true for the output of SoapySDRUtil --args='driver=sdrplay,biasT_ctrl=false' --rate=8e6 --direction=RX. Again, no indication of an error.

######################################################
##     Soapy SDR -- the SDR abstraction library     ##
######################################################

Stream format: CS16
Num channels: 1
Element size: 4 bytes
Begin RX rate test at 8 Msps
Starting stream loop, press Ctrl+C to exit...
7.42988 Msps    29.7195 MBps
7.7193 Msps     30.8772 MBps
7.81245 Msps    31.2498 MBps
7.85368 Msps    31.4147 MBps
7.88555 Msps    31.5422 MBps
7.90561 Msps    31.6224 MBps
7.91864 Msps    31.6746 MBps
7.92827 Msps    31.7131 MBps
[... many more such lines - but no errors ... ]
7.95986 Msps    31.8394 MBps
7.96195 Msps    31.8478 MBps
7.96338 Msps    31.8535 MBps
7.96647 Msps    31.8659 MBps
7.96883 Msps    31.8753 MBps
7.96993 Msps    31.8797 MBps

Have I overlooked something in my procedure?

thowe-switzerland commented 3 years ago

I noticed that I have much less log output with the unstable branch of RTLSDR-Airband than with the version from master branch version. Since I have not found out how to increase the number of log messages, I have installed without further ado again the version from the master branch, which flies off the track at the first overflow and stops, but at least logs more.

And as expected, I now have the log file of a run until the overflow occurred - and this version of RTLSDR airband turned off. From start to overflow/stop it took about 12 minutes. But the runtime between two overflows I observed is normally between 7 and 17 minutes.

First we see one log with overflow #2 and then a lot with overflow #1.

pi@atc-stream-rpi:~/RTLSDR-Airband-3.2.1 $ sudo rtl_airband -Fe
[INFO] devIdx: 0
[INFO] hwVer: 255
[INFO] rspDuoMode: 0
[INFO] tuner: 1
[INFO] rspDuoSampleFreq: 0.000000
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true': using native sample format 'CS16' (fullScale=32767.0)
SoapySDR: device 'driver=airspy': using native sample format 'CS16' (fullScale=32767.0)
RTLSDR-Airband version 3.1.0 starting
[INFO] devIdx: 0
[INFO] hwVer: 255
[INFO] rspDuoMode: 0
[INFO] tuner: 1
[INFO] rspDuoSampleFreq: 0.000000
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true': sample rate set to 7000000 sps
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true': AGC off (requested: off)
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true': gain 'IFGR' set to 30.0 dB
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true': gain 'RFGR' set to 8.0 dB
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true' initialized
[INFO] Using format CS16.
SoapySDR: device 'driver=airspy': sample rate set to 3000000 sps
SoapySDR: device 'driver=airspy': AGC off (requested: off)
SoapySDR: device 'driver=airspy': gain 'LNA' set to 4.0 dB
SoapySDR: device 'driver=airspy': gain 'MIX' set to 4.0 dB
SoapySDR: device 'driver=airspy': gain 'VGA' set to 10.0 dB
SoapySDR: device 'driver=airspy' initialized
[INFO] Using format CS16.
SoapySDR: device 'driver=airspy' started
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true' started
Writing to /home/pi/rtlsdr_airband_data/twr_mp3//TWR_RSP1A_20210530_15.mp3
Writing to /home/pi/rtlsdr_airband_data/twr_mp3//TWR_AIRSPY_20210530_15.mp3
[WARNING] Sun May 30 17:12:13 2021 - overflow #2
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
[WARNING] Sun May 30 17:12:13 2021 - overflow #1
OSoapySDR device 'driver=sdrplay,biasT_ctrl=true': readStream failed: OVERFLOW, disabling
Disabling device output
fventuri commented 3 years ago

@thowe-switzerland - Thomas, thanks for the detailed reports and the very thorough analysis.

I am not familiar at all with the program RTLSDR-Airband; that's why I am asking you to also run tests using the stream loop test from SoapySDRUtil, since if we can reproduce the overflow problem with this utility, it becomes much easier for me to troubleshoot it.

I just pushed a new commit (https://github.com/pothosware/SoapySDRPlay3/commit/9d1c10d34a166a07b132892df68c2f52c34880f2) to the branch debug_overflows; since the SDRplay API also returns the sequence number of the first sample when calling the RX streaming callback, I added a simple check to make sure there are no gaps in the sequence numbers (i.e. that each callback starts with a sequence number that is exactly after the last sequence number of the previous callback). I just run a test using SoapySDRUtil --args='driver=sdrplay' --rate=8e6 --direction=RX for more than 30 minutes with the RSPdx I have here, and it didn't report any error.

Please build the SoapySDRPlay driver again using the latest code from the debug_overflows branch; run the usual tests with both RTLSDR-Airband and SoapySDRUtil (I think 30 minutes or so for each test should be enough at this point), and let me know if you see any warning due to gaps in the sample sequence numbers.

Franco

thowe-switzerland commented 3 years ago

Using the new version rtl_airband is stopped with the expected additional log entry. You find the complete output below.

However, the RX tests with SoapySDRUtil --args='driver=sdrplay' --rate=8e6 --direction=RX run without problems for more than 60 minutes - without any warning. I even used all CPUs in parallel with stress-ng - without being able to provoke the problem that way... I refrain from inserting the error-free output here ;-)

pi@atc-stream-rpi:~/RTLSDR-Airband-3.2.1 $ sudo rtl_airband -Fe
[INFO] devIdx: 0
[INFO] hwVer: 255
[INFO] rspDuoMode: 0
[INFO] tuner: 1
[INFO] rspDuoSampleFreq: 0.000000
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true': using native sample format 'CS16' (fullScale=32767.0)
SoapySDR: device 'driver=airspy': using native sample format 'CS16' (fullScale=32767.0)
RTLSDR-Airband version 3.1.0 starting
[INFO] devIdx: 0
[INFO] hwVer: 255
[INFO] rspDuoMode: 0
[INFO] tuner: 1
[INFO] rspDuoSampleFreq: 0.000000
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true': sample rate set to 7000000 sps
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true': AGC off (requested: off)
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true': gain 'IFGR' set to 30.0 dB
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true': gain 'RFGR' set to 8.0 dB
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true' initialized
[INFO] Using format CS16.
SoapySDR: device 'driver=airspy': sample rate set to 3000000 sps
SoapySDR: device 'driver=airspy': AGC off (requested: off)
SoapySDR: device 'driver=airspy': gain 'LNA' set to 4.0 dB
SoapySDR: device 'driver=airspy': gain 'MIX' set to 4.0 dB
SoapySDR: device 'driver=airspy': gain 'VGA' set to 10.0 dB
SoapySDR: device 'driver=airspy' initialized
[INFO] Using format CS16.
SoapySDR: device 'driver=airspy' started
SoapySDR: device 'driver=sdrplay,biasT_ctrl=true' started
Writing to /home/pi/rtlsdr_airband_data/twr_mp3//TWR_AIRSPY_20210530_20.mp3
Writing to /home/pi/rtlsdr_airband_data/twr_mp3//TWR_RSP1A_20210530_20.mp3
[WARNING] Sun May 30 22:47:45 2021 - sample num gap in stream: 3162541202 [3162541206:0] -> 4+631636000
[WARNING] Sun May 30 22:56:50 2021 - overflow #2
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:50 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
[WARNING] Sun May 30 22:56:51 2021 - overflow #1
O[WARNING] Sun May 30 22:56:51 2021 - returning SOAPY_SDR_OVERFLOW
SoapySDR device 'driver=sdrplay,biasT_ctrl=true': readStream failed: OVERFLOW, disabling
Disabling device outputWriting to /home/pi/rtlsdr_airband_data/twr_mp3//TWR_AIRSPY_20210530_21.mp3
fventuri commented 3 years ago

@thowe-switzerland - Thomas, thanks again for all your time and patience researching this problem.

Given the fact that the SoapySDRUtil stream loop test is not reporting any warnings, I am starting to wonder if you are seeing the problem described by @szpajder in this comment in the thread you sent me yesterday: https://github.com/szpajder/RTLSDR-Airband/issues/247#issuecomment-842503241 - in other words that there are occasional CPU load spikes (perhaps due to other processes, or maybe simply caused by OS background tasks, since I don't think you are using a RT OS there), and the CPU Is not able to keep up with the 8Msps stream coming from the RSP1.

As @szpajder suggested, I would try to run a few tests with the RSP1 at lower sample rates (say 4MHz and 6 MHz), to see if the overflow problem still happens at those rates; also at this point, for the SoapySDRPlay driver you may want to go back to building and running the code from the master branch for SoapySDRPlay, because I don't think the extra debug statements in the debug_overflows branch will tell us anything we don't already know.

Franco

SDRplay commented 3 years ago

Couple of questions...

What RPi is this being run on? What happens when you just run the RSP by itself?

In our testing the RPi can handle up to 10 MHz sample rate with the RSP - however we don't test with more than one SDR at a time.

Best regards,

Andy

thowe-switzerland commented 3 years ago

Hi Andy

Thanks for hanging in there too.

What RPi is this being run on?

Raspberry Pi 4B 4GB

What happens when you just run the RSP by itself?

The same error behavior when only the RSP1A is attached and in use. I had just recently added the Airspy to compare the resulting audio file.

Thanks and best regards, Thomas

thowe-switzerland commented 3 years ago

Regarding the utilization of the RPi 4 in the setup, I can say that the 4 cores are pretty consistently utilized around 40% to 60%. I don't see any bottleneck with htop, but I would have to take a closer look.

I have also tried with SoapySDRUtil --args='driver=sdrplay' --rate=8e6 --direction=RX and simultaneous stress-ng tests to deliberately cause an overload. Without the problem occurring. Of course, I can't rule out the hardware or its load behavior. But it is at least nothing trivial (for me).

I will now do some tests with the RSP1A by reducing the load: without other devices, smaller sample rates. I will report back as soon as I can make new statements.

In the meantime I want to thank you all for the cool software and the great support here!

thowe-switzerland commented 3 years ago

I have now done extensive experiments with the Raspberry Pi 4 B, the RSP1A and the Airspy mini via SoapySDR.

It is indeed the case that overflows become more likely as bandwidth increases:

The Airspy mini seems to behave similarly. Since it only supports 3.0, 6.0 and 10.0 MSPS, I can only judge these. With this one, the overflows at 10 MSPS occur after about 10 minutes.

I additionally tried to stress the CPUs with stress-ng (e.g. load ~3.80 with 4 cores). The effect on the frequency of overflows seems weak. This is actually astonishing. My gut feeling is that the cause for the overflows is probably not overloadeded CPU cores. It must be something else.

I'll see if I can get a real-time kernel running and see if the timing is more consistent over time with it. And if it can prevent these Overflows.

fventuri commented 3 years ago

@thowe-switzerland - Thomas, thanks for your extensive experiments and for reporting your very interesting findings!

I did some quick research in these days (sorry for not replying to you earlier, but during the week I can only spend a couple of hours of my free time after work/dinner on this kind of project), and I found a couple of interesting pages where they talk about real time issues for audio and music related applications; your case is slightly different, but I thought some of their comments and suggestions might apply to streaming off the RSP1A (and also if any of these ideas work, they might save you from having to patch, rebuild, run, and possibly troubleshoot, a real time kernel).

The first one is this tool written in Perl called 'realtimequickconfigscan', that runs a quick check of your Linux installation for potential causes of real time streaming issues: https://github.com/raboof/realtimeconfigquickscan

The second one is this section of the Fedora page about the audio application 'JACK' - https://fedoraproject.org/wiki/JACK_Audio_Connection_Kit#Running_Jack_in_Realtime_mode - they have some commands on how to make a process (RTLSDR-Airband in your case) run with real time priority; they also show there what the output from 'realtimeconfigquickscan' looks like for a well configured system, so you can see how to possibly address those short skips you noticed.

Franco

thowe-switzerland commented 3 years ago

Hello Franco

Thanks for the support and the very good tips. There is no need at all for you to apologize. I am thrilled with the software and the involvement here. Clearly we all have other commitments....

Effectively, I have now already built an RT kernel. Since there is some risk that it will not work, I like to try the optimizations you mentioned first.

I think the RPi 4 actually has enough power to handle the 8 MSPS. The CPU load shows that. The USB bus is also quite powerful now. Over (relatively) long periods of time, streaming works flawlessly. So it could effectively be a timing issue, which I'm still getting to grips with.

I'll get back to you as soon as I have news.

thowe-switzerland commented 3 years ago

I did the optimization of the Raspberry Pi OS according to the referenced Pearl script. At the end almost everything was "green". Unfortunately without noticeable effect on the "hiccups" that still happened.

What I have now done today: Installed the whole system based on Ubuntu Server 64 bit for Raspberry Pi.

The system is now running for an hour with 10 MSPS - so far without any problems.

If this continues, it would mean that it could be a problem only on Raspberry Pi OS Lite (and maybe even only this specific version).

I'm continuing my tests now and sticking with Ubuntu Server 64 bit for now.

thowe-switzerland commented 3 years ago

I did further extensive testing and made the following observations:

The current Raspberry Pi OS Lite 32 bit for Rasperry Pi 4 B (all standard unmodified or optimized for audio load) runs error free with SoapySDR at 7 MSPS for about 0.5h. Overflows occur after about this time.

The current Ubuntu Server 64 bit for Rasperry Pi 4 B (all standard unchanged) runs with SoapySDR at 7 MSPS without errors for around 2.5h. Overflows also occur after about this time. Roughly speaking, overflows are about 5 times less frequent than with standard Raspberry Pi OS Lite.

The current Raspberry Pi OS Lite 32 Bit for Rasperry Pi 4 B (all standard unchanged but with self compiled PREEMPT-RT kernel 5.10.39-rt39-v7l+) runs with SoapySDR at 7 MSPS error free for about 2.5h. Overflows occur after about this time. Roughly speaking, overflows are about 5 times less frequent than with standard Raspberry Pi OS Lite.

Whoever reads this should keep in mind that these are snapshots. It is possible that this will change with the next version of Raspberry Pi OS or that others will have completely different experiences.

fventuri commented 3 years ago

@thowe-switzerland - thanks again for the very detailed research you did these days. Just to rule out that the issue is not caused by the delay in writing the mp3 file to the SD card (if that is the way RTLSDR-Airband works), would it be possible to run a test where the output from RTLSDR-Airband goes to /dev/null, to see if the overflows happen in that case too?

Franco

thowe-switzerland commented 3 years ago

Thanks Franco. Good point. I could indeed take a closer look at the file system.

I tried to specify dev/null as directory for the generated MP3 files. Unfortunately this leads to the error message that this is not a directory. I have found so far no possibility, that the MP3 file is written efficiently into the Nirvana.

In the end I will use mainly Icecast streams (for LiveATC feed) and C16 files (for ATCO2 project). A local MP3 file would be at most for the easy local quality control of advantage.

thowe-switzerland commented 3 years ago

Probably we come to a temporary end with this ticket for several reasons.

With the specific choice of OS (Ubuntu Server or Raspberry Pi OS with RT kernel), I now have a sufficiently stable system. I can probably deal with one buffer overflow per approx. 2h.

On the other hand, my tests have shown:

I learned a lot. From my perspective, this ticket can be closed here. I would like to thank you from the bottom of my heart for the great helpfulness! Cool software. Cool people behind it.

fventuri commented 3 years ago

@thowe-switzerland - thanks for your very nice words Thomas! I too have learned a lot in the process of troubleshooting this issue, and I really appreciate your effort and diligence in researching the possible causes and looking for solutions.

I am going to close this issue now, but if you have more findings and perhaps figure out how to 'tame' all the different components (hw, os, application, etc) to make it stream without buffer overflows, I (and others who might have a similar experience in the future) would be very interested in hearing any updates, so feel free to add to this discussion, or to create a new one.

Thanks again, Franco