RTLSDR: buffer overrun

[Taxom]

I encountered the problem "aiscatcher[3028972]: RTLSDR: buffer overrun" which seems to me to occur when connecting to ais-catcher via TCP on open port 5011.

droped CPU use: but memory enough stops sending data

[Taxom]

looked at the statistics, and not sure that this only happens when an open TCP port is used. The first time this happened was a couple of weeks ago, before I opened the TCP port. But when I start connecting via TCP just by opening and closing it from the browser (IP_Pi:5011) in several tabs, the issue almost always reproduces.

[jvde-github]

Hi, thanks!

Can you share the model settings you are using? Run 'top' on the device and look up the cpu load of AIS-catcher and share some more details on the device?

The model settings are printed in the log, interested in sample rate, buffer count, etc and version.

Thanks, Jasper

[Taxom]

My settings:

-d 00000162 -v 10 -M DT -gr RTLAGC off TUNER 26 -m 4 -go AFC_WIDE on -a 100K -s 1536k -p -2 -o 5 filter off block_type 1,2,4,5,9,18,19,21 -u 127.0.0.1 10111 -S 5011 -u 192.168.1.145 10103 JSON on -u 192.168.1.100 10101 -u 5.9.207....... -u 195.201.71...... -u 144.76.54.1...... -u 31.172.90.2...... -u 144.76.105...... -u 178.162.215.1....... -u 148.251.9....... -N 8383 STYLE colors.txt FILE stat.bin BACKUP 60 ABOUT about_test.md MESSAGE on -N HISTORY 3600 -N STATION MaxPi_AIS_Toronto STATION_LINK https://www.myshiptracking.com/stations/station?id=736 -N PLUGIN_DIR /usr/share/aiscatcher/plugins -N LAT 43.6 LON -79.4 share_loc on -N REUSE_PORT on

top: nmon:

system:

hardware: Orange Pi Zero 2 http://www.orangepi.org/html/hardWare/computerAndMicrocontrollers/details/Orange-Pi-Zero-2.html 2 RTL-SDR for AIS and ADS-B connected by wifi 5

[Taxom]

This problem also occurs if I connect to it from the AIS-catcher-android mobile application via TCP (on 5011) when the smartphone turns off the screen.

[jvde-github]

Ok all good. Seems the program is running quite comfortably. So not a performance issue with the hardware.

With the last example you mean that you see the AIS-catcher instance on the Pi stop sending messages if the Android phone turns off the screen?

I will have a think about the link with TCP, but any other hints that can help from the log file? Is this something that occurs if and only if you connect to this TCP listener at 5011? In particular, I notice a long disruption at 10pm. You were running a TCP client around this time?

The messages for this TCP listener are send from the main decoding thread (in a non-blocking way) so perhaps something is delaying which still blocks the decoder. Need to explore this angle.

[Taxom]

Yes, at about 22:00, while walking, I connected from my phone (ais-catcher_android) to my home ais-catcher (on Orange-Pi). When I got home, I saw this problem in the statistics of one of the sites that I feed, and this prompted me to the fact that the problem is related to TCP. all further breaks are connected with my attempts to repeat the problem.

"The messages for this TCP listener are send from the main decoding thread (in a non-blocking way) so perhaps something is delaying which still blocks the decoder. Need to explore this angle."

Maybe it's worth putting it in a separate thread so that it can be processed by a separate processor core ?!

[Taxom]

Just now I will repeat the problem again by connecting from my smartphone (ais-catcher_android) and leaving it to work. At the same time, it is clearly seen that ais-catcher on orange-pi stops loading the processor.

[Taxom]

Moreover, the work was resumed not when I pressed the stop button in the ais-catcher-androd application, but when I forcibly closed the application on the phone.

[Taxom]

[Taxom]

At the same time, I really like the ability to connect via TCP, as this makes it possible to use OpenCPN on a remote phone (tablet), as well as ais-catcher_andriod.

[jvde-github]

Great, this suggests it is the SendAll that is blocking the system (stops decoding so lower CPU usage).

I don't understand why it would block but have an idea how to solve it (move it to the Server thread) so it does not stop the main thread.

The force stop issue on Android to close a TCP connection has been resolved today 😁

[jvde-github]

Ok, I created an experimental branche listener. Could you try if your use this branche the problem that you experience is resolved. This would show we are on the right track. The build instruction is now slightly differently in cloning:

git clone https://github.com/jvde-github/AIS-catcher.git -b listener

[Taxom]

Updated:

root@OPiZ2:~# sudo bash -c "$(wget -O - https://raw.githubusercontent.com/Taxom/install-aiscatcher/5f026ea9e24bddd98dd49e41fd0ca762122e86b3/install-aiscatcher.sh)" --2023-08-27 16:35:33-- https://raw.githubusercontent.com/Taxom/install-aiscatcher/5f026ea9e24bddd98dd49e41fd0ca762122e86b3/install-aiscatcher.sh Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 2606:50c0:8003::154, 2606:50c0:8002::154, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 6809 (6.6K) [text/plain] Saving to: ‘STDOUT’

• 100%[====================================================================================================>] 6.65K --.-KB/s in 0.002s

2023-08-27 16:35:34 (3.75 MB/s) - written to stdout [6809/6809]

Installing build tools and dependencies... Reading package lists... Done Building dependency tree... Done Reading state information... Done git is already the newest version (1:2.30.2-1+deb11u2). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Reading package lists... Done Building dependency tree... Done Reading state information... Done make is already the newest version (4.3-4.1). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Reading package lists... Done Building dependency tree... Done Reading state information... Done gcc is already the newest version (4:10.2.1-1). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Reading package lists... Done Building dependency tree... Done Reading state information... Done g++ is already the newest version (4:10.2.1-1). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Reading package lists... Done Building dependency tree... Done Reading state information... Done cmake is already the newest version (3.18.4-2+deb11u1). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Reading package lists... Done Building dependency tree... Done Reading state information... Done pkg-config is already the newest version (0.29.2-1). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Reading package lists... Done Building dependency tree... Done Reading state information... Done librtlsdr-dev is already the newest version (0.6.0-3). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Reading package lists... Done Building dependency tree... Done Reading state information... Done whiptail is already the newest version (0.52.21-4+b3). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Reading package lists... Done Building dependency tree... Done Reading state information... Done libpq-dev is already the newest version (13.10-0+deb11u1). 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. Creating folder aiscatcher if it does not exist Creating startup script file start-ais.sh Writing code to startup script file start-ais.sh Creating Service file aiscatcher.service Entering install folder... code exists Updating code Fetching origin remote: Enumerating objects: 8, done. remote: Counting objects: 100% (8/8), done. remote: Compressing objects: 100% (8/8), done. remote: Total 8 (delta 2), reused 0 (delta 0), pack-reused 0 Unpacking objects: 100% (8/8), 7.06 KiB | 516.00 KiB/s, done. From https://github.com/jvde-github/AIS-catcher

• [new branch] listener -> origin/listener HEAD is now at f88c0db Stamp version and bake index.html in HTML.cpp Making executable binary -- The C compiler identification is GNU 10.2.1 -- The CXX compiler identification is GNU 10.2.1 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.2") -- Looking for pthread.h -- Looking for pthread.h - found -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE -- Checking for module 'libusb-1.0' -- Found libusb-1.0, version 1.0.24 -- Checking for module 'libcurl' -- No package 'libcurl' found -- CURL: not found - CURL_INCLUDE_DIR-NOTFOUND, CURL_LIBRARY-NOTFOUND -- Checking for module 'zlib' -- Found zlib, version 1.2.11 -- ZLIB: found - /usr/include, /usr/lib/aarch64-linux-gnu/libz.so -- Checking for module 'libpq' -- Found libpq, version 13.10 -- PQ: found - /usr/include/postgresql, /usr/lib/aarch64-linux-gnu/libpq.so -- Checking for module 'soxr' -- No package 'soxr' found -- SOXR: not found. -- Checking for module 'samplerate' -- No package 'samplerate' found -- libsamplerate: not found. -- Checking for module 'librtlsdr' -- Found librtlsdr, version 0.6.0 -- RTLSDR: found - /usr/include, /usr/lib/aarch64-linux-gnu/librtlsdr.so -- Looking for rtlsdr_set_bias_tee -- Looking for rtlsdr_set_bias_tee - found -- RTLSDR: bias-tee support included. -- Looking for rtlsdr_set_tuner_bandwidth -- Looking for rtlsdr_set_tuner_bandwidth - found -- RTLSDR: tuner badwidth support included. -- SDRPLAY: not found. -- Checking for module 'libairspy' -- No package 'libairspy' found -- AIRSPY: not found. -- Checking for module 'libairspyhf' -- No package 'libairspyhf' found -- AIRSPYHF: not found. -- Checking for module 'libhackrf' -- Found libhackrf, version 0.5 -- HACKRF: found - /usr/include/libhackrf, /usr/lib/aarch64-linux-gnu/libhackrf.so -- Checking for module 'libzmq' -- No package 'libzmq' found -- ZMQ: not found. -- Configuring done -- Generating done -- Build files have been written to: /usr/share/aiscatcher/AIS-catcher/build Scanning dependencies of target AIS-catcher [ 2%] Building CXX object CMakeFiles/AIS-catcher.dir/Application/Main.cpp.o [ 5%] Building CXX object CMakeFiles/AIS-catcher.dir/Application/Receiver.cpp.o [ 8%] Building CXX object CMakeFiles/AIS-catcher.dir/Application/Config.cpp.o [ 11%] Building CXX object CMakeFiles/AIS-catcher.dir/Ships/DB.cpp.o [ 14%] Building CXX object CMakeFiles/AIS-catcher.dir/DBMS/PostgreSQL.cpp.o [ 17%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/AIRSPYHF.cpp.o [ 20%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/FileWAV.cpp.o [ 22%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/RTLSDR.cpp.o [ 25%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/SDRPLAY.cpp.o [ 28%] Building CXX object CMakeFiles/AIS-catcher.dir/DSP/Demod.cpp.o [ 31%] Building CXX object CMakeFiles/AIS-catcher.dir/DSP/Model.cpp.o [ 34%] Building CXX object CMakeFiles/AIS-catcher.dir/Library/AIS.cpp.o [ 37%] Building CXX object CMakeFiles/AIS-catcher.dir/Library/JSONAIS.cpp.o [ 40%] Building CXX object CMakeFiles/AIS-catcher.dir/Library/Keys.cpp.o [ 42%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/FileRAW.cpp.o [ 45%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/HACKRF.cpp.o [ 48%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/UDP.cpp.o [ 51%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/RTLTCP.cpp.o [ 54%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/ZMQ.cpp.o [ 57%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/SoapySDR.cpp.o [ 60%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/SpyServer.cpp.o [ 62%] Building CXX object CMakeFiles/AIS-catcher.dir/Library/Message.cpp.o [ 65%] Building CXX object CMakeFiles/AIS-catcher.dir/Library/NMEA.cpp.o [ 68%] Building CXX object CMakeFiles/AIS-catcher.dir/Library/Utilities.cpp.o [ 71%] Building CXX object CMakeFiles/AIS-catcher.dir/Library/TCP.cpp.o [ 74%] Building CXX object CMakeFiles/AIS-catcher.dir/JSON/JSON.cpp.o [ 77%] Building CXX object CMakeFiles/AIS-catcher.dir/IO/Network.cpp.o [ 80%] Building CXX object CMakeFiles/AIS-catcher.dir/IO/HTTPServer.cpp.o [ 82%] Building CXX object CMakeFiles/AIS-catcher.dir/JSON/StringBuilder.cpp.o [ 85%] Building CXX object CMakeFiles/AIS-catcher.dir/JSON/Parser.cpp.o [ 88%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/AIRSPY.cpp.o [ 91%] Building CXX object CMakeFiles/AIS-catcher.dir/Device/Serial.cpp.o [ 94%] Building CXX object CMakeFiles/AIS-catcher.dir/DSP/DSP.cpp.o [ 97%] Building CXX object CMakeFiles/AIS-catcher.dir/IO/IO.cpp.o [100%] Linking CXX executable AIS-catcher [100%] Built target AIS-catcher Copying AIS-catcher binary in folder /usr/local/bin/ Stoping existing aiscatcher to enable over-write Copying newly built binary "AIS-catcher" to folder "/usr/local/bin/" Renaming existing folder "my-plugins" to "my-plugins.old" Copying files from Source code folder "AIS-catcher/plugins" to folder "my-plugins" User aiscat already exists. Not creating it again Assigning ownership of install folder to user aiscat

INSTALLATION COMPLETED

PLEASE DO FOLLOWING:

(1) If on RPi you have installed AIS Dispatcher or OpenCPN, it should be configured to use UDP Port 10110, IP 127.0.0.1 OR 0.0.0.0 (2) Open file aiscatcher.conf by following command: sudo nano /usr/share/aiscatcher/aiscatcher.conf (3) In above file: (a) Change 00000162 in "-d 00000162" to actual Serial Number of AIS dongle (b) Change 3 in "-p 3" to the actual ppm correction figure of dongle (c) Change 38.6 in "-gr TUNER 38.6 RTLAGC off" to desired Gain of dongle (d) Add following line and replace xx.xxx and yy.yyy by actual values: -N STATION MyStation LAT xx.xxx LON yy.yyy (e) For each Site you want to feed AIS data, add a new line as follows: -u [URL or IP of Site] [Port Number of Site] (f) Save (Ctrl+o) and Close (Ctrl+x) file aiscatcher.conf

IMPORTANT: If you are Upgrading or Reinstalling,your old config file & pluin folder are saved as /usr/share/aiscatcher/aiscatcher.conf.old /usr/share/aiscatcher/my-plugins.old

(4) REBOOT RPi ... REBOOT RPi ... REBOOT RPi

(5) See the Web Interface (Map etc) at 192.168.1.155:8383 (IP-of-PI:8383)

(6) Command to see Status sudo systemctl status aiscatcher (7) Command to Restart sudo systemctl restart aiscatcher root@OPiZ2:~# systemctl status aiscatcher ● aiscatcher.service - AIS-catcher Loaded: loaded (/lib/systemd/system/aiscatcher.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2023-08-27 16:44:24 EDT; 4min 37s ago Main PID: 4108584 (bash) Tasks: 8 (limit: 1012) Memory: 12.1M CPU: 43.939s CGroup: /system.slice/aiscatcher.service ├─4108584 /bin/bash /usr/share/aiscatcher/start-ais.sh └─4108585 /usr/local/bin/AIS-catcher -d 00000162 -v 10 -M DT -gr RTLAGC off TUNER 26 -m 4 -go AFC_WIDE on -a 100K -s 1536k -p -2 -o 5 filter off block_type 1,2,4,5,9,18,19,21 >

Aug 27 16:49:00 OPiZ2 aiscatcher[4108585]: {"class":"AIS","device":"AIS-catcher","rxtime":"20230827204900","scaled":true,"channel":"B","nmea":["!AIVDM,1,1,,B,24eI1NP001rENgHHdJtjq5Mn0D0a,0> Aug 27 16:49:01 OPiZ2 aiscatcher[4108585]: {"class":"AIS","device":"AIS-catcher","rxtime":"20230827204901","scaled":true,"channel":"B","nmea":["!AIVDM,1,1,,B,34eHd6MPA=JDUjfHv1=4aCUn00r0,0> Aug 27 16:49:01 OPiZ2 aiscatcher[4108585]: {"class":"AIS","device":"AIS-catcher","rxtime":"20230827204901","scaled":true,"channel":"A","nmea":["!AIVDM,1,1,,A,24eGlgP000rENWlHc;K0g0F00PS=,0> Aug 27 16:49:01 OPiZ2 aiscatcher[4108585]: {"class":"AIS","device":"AIS-catcher","rxtime":"20230827204901","scaled":true,"channel":"B","nmea":["!AIVDM,1,1,,B,14eGmP@000JCU1JHpK?eA9p006S4,0> Aug 27 16:49:01 OPiZ2 aiscatcher[4108585]: {"class":"AIS","device":"AIS-catcher","rxtime":"20230827204901","scaled":true,"channel":"A","nmea":["!AIVDM,1,1,,A,4030p2iu@Eli0rC1UBHplaG0080t,0> Aug 27 16:49:01 OPiZ2 aiscatcher[4108585]: {"class":"AIS","device":"AIS-catcher","rxtime":"20230827204901","scaled":true,"channel":"B","nmea":["!AIVDM,1,1,,B,23ku4VP000JENu6HcRD=25f00L0v,0> Aug 27 16:49:01 OPiZ2 aiscatcher[4108585]: {"class":"AIS","device":"AIS-catcher","rxtime":"20230827204901","scaled":true,"channel":"A","nmea":["!AIVDM,1,1,,A,24eICv@00SrEGoBHgHFo7Ef00<0j,0> Aug 27 16:49:02 OPiZ2 aiscatcher[4108585]: {"class":"AIS","device":"AIS-catcher","rxtime":"20230827204902","scaled":true,"channel":"A","nmea":["!AIVDM,1,1,,A,4030owivN=li1rDt7BH`aPG00H1<,0> Aug 27 16:49:02 OPiZ2 aiscatcher[4108585]: {"class":"AIS","device":"AIS-catcher","rxtime":"20230827204902","scaled":true,"channel":"B","nmea":["!AIVDM,1,1,,B,34eJ;q0P01rDSshHuqFN4?v224hS,0> Aug 27 16:49:02 OPiZ2 aiscatcher[4108585]: {"class":"AIS","device":"AIS-catcher","rxtime":"20230827204902","scaled":true,"channel":"A","nmea":["!AIVDM,1,1,,A,33ku:aU000rBN22Hhl@TCkB00DSb,0>

I hope I did it right, because the version is displayed the same

and the problem hasn't gone away

maybe i did something wrong during update

[jvde-github]

I made a small tweak so that version is shown as listener in webclient and just uploaded that. Then we are sure you have the right version. I am a little confused this did not resolve it. I might have see if I can replicate the problem myself....

[Taxom]

Sorry, I can't update to this thread. I tried to do this using the script "https://github.com/abcd567a/install-aiscatcher" replacing the line "git clone https://github.com/jvde-github/AIS-catcher.git" with "git clone https://github.com/jvde-github/AIS-catcher.git -b listener", but that didn't seem to work. (((

[Taxom]

I can update:

but received 200% usage CPU

[Taxom]

But the initial problem seems to be solved.

[Taxom]

I had to roll back because 200% on CPU was too much.

[jvde-github]

This confirms that indeed we found the root cause for the idling. I also fixed the CPU load (in the listener branche again and you can see the designation listener 2 in the webclient). So please try again. In particular the CPU load as well in case you reproduce this idle state situation.

How did you trigger the idle state? Via the Android app and then it let turn of screen?

I will test a bit more in the coming week and then roll it out into production. Thanks! I am not able to reproduce this btw but I saw a problem in the code that fitted with your observations.

[jvde-github]

Hi! I have just merged the whole listener branche into the production branche. Fingers crossed :-) Hopefully this has resolved your issues and with usual processor load.

Thanks, Jasper

[Taxom]

I launched the android application, pressed start in it, and then forcibly turned off the screen on phone. This with a probability of 80-90% led to the buffer overrun.

I upgraded to the "listener 2" branch in the morning and after three hours of trying to repeat the problem, it no longer occurred.

The only thing I noticed is a slightly higher memory consumption of 27MB instead of 19MB, but this is not a problem if everything works.

[Taxom]

[Taxom]

sorry for not replying earlier

[jvde-github]

Well... thank you for excellent troubleshooting. Without your clear observations would have been impossible to fix this.

[Taxom]

I'm happy to be helpful! Thanks for all you are doing!