jvde-github / AIS-catcher

AIS receiver for RTL SDR dongles, Airspy R2, Airspy Mini, Airspy HF+, HackRF, SDRplay and SoapySDR
https://aiscatcher.org
GNU General Public License v3.0
470 stars 76 forks source link

`libcurl` missing for HTTP use #33

Closed kx1t closed 1 year ago

kx1t commented 1 year ago

I built the package from source using your instructions and included only the RTLSDR drivers. Trying to use it to feed aprs.fi, and I', seeing this (redacted for privacy):

$ /usr/bin/AIS-catcher -H http://aprs.fi/jsonais/post//jsonais/post/z...........V kx1t PROTOCOL aprs INTERVAL 30 -u 127.0.0.1 34995 -u 5.9.207.224 1...3 -u ais.vesselfinder.com 5...9 -q -v 60 -d AIS -gr tuner 49.6 rtlagc ON
HTTP: not implemented, please recompile with libcurl support.

I'm assuming I need to rebuild with libcurl-dev installed -- but it wants me to explicitly select a package. Which one should I use? Or am I doing something else wrong?

# apt install libcurl-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Package libcurl-dev is a virtual package provided by:
  libcurl4-openssl-dev 7.74.0-1.3+deb11u3
  libcurl4-nss-dev 7.74.0-1.3+deb11u3
  libcurl4-gnutls-dev 7.74.0-1.3+deb11u3
You should explicitly select one to install.

(Bedankt voor al het werk en de tijd die je hierin hebt geinvesteerd! - Ramon kx1t@amsat.org)

kx1t commented 1 year ago

Hmmm nevermind - I think I see you add libcurl4-openssl-dev yourself in your Dockerfile. I'll try it out and close the Issue if that works for me.

jvde-github commented 1 year ago

Hi Ramon, yes indeed, I don't think it makes a big difference which version you install. Hope this works and thanks for trying this new function :-)

jvde-github commented 1 year ago

One observation, your APRS url looks off:

http://aprs.fi/jsonais/post/**/jsonais/post/**z...........V

Should it be?

http://aprs.fi/jsonais/post/z...........V

kx1t commented 1 year ago

Yeah -- I have it somewhat working now, but I am getting "HTTP Out of memory" errors:

/usr/bin/AIS-catcher -u 127.0.0.1 3....5 -u 5.9.207.224 1...3 -u ais.vesselfinder.com 5..9
 -q -v 60 -d AIS -gr tuner 49.6 rtlagc ON -H http://aprs.fi/jsonais/post/z.........V ID kx1t PROTOCOL aprs INTERVAL 30

Logs output:

[aiscatcher][2022/11/07 16:35:25] AIS-catcher (build Nov  7 2022) v0.39
[aiscatcher][2022/11/07 16:35:25] (C) Copyright 2021-2022 jvde-github and other contributors
[aiscatcher][2022/11/07 16:35:25] This is free software; see the source for copying conditions.There is NO
[aiscatcher][2022/11/07 16:35:25] warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
[aiscatcher][2022/11/07 16:35:25] Device selected: Realtek, RTL2838UHIDIR, SN: AIS
[aiscatcher][2022/11/07 16:35:25] Found Rafael Micro R820T tuner
[aiscatcher][2022/11/07 16:35:25] Device: RTL2838UHIDIR
[aiscatcher][2022/11/07 16:35:25] Settings: rate 1536K tuner 49.599998 rtlagc ON biastee OFF
[aiscatcher][2022/11/07 16:35:25] HTTP: start server (http://aprs.fi/jsonais/post/z........V).
[aiscatcher][2022/11/07 16:35:25] [R82XX] PLL not locked!
[aiscatcher][2022/11/07 16:35:25] Allocating 15 zero-copy buffers
...
[aiscatcher][2022/11/07 16:35:56] HTTP: error - Out of memory (http://aprs.fi/jsonais/post/z..........V)
[aiscatcher][2022/11/07 16:35:56] HTTP: server response - 
[aiscatcher][2022/11/07 16:36:25] [AIS engine v0.39]                      received: 187 msgs, total: 187 msgs, rate: 3.10253 msg/s
[aiscatcher][2022/11/07 16:36:26] HTTP: error - Out of memory (http://aprs.fi/jsonais/post/z..........V)
[aiscatcher][2022/11/07 16:36:26] HTTP: server response - 
...

(Note - I am running this inside a Docker container, debian-slim base, Raspi 4B/4Gb, and htop shows that I have plenty resources left) image

Not sure if it helps, but this is what's installed in my container:

# apt-cache policy libcurl*
libcurl3:
  Installed: (none)
  Candidate: (none)
  Version table:
libcurl4:
  Installed: 7.74.0-1.3+deb11u3
  Candidate: 7.74.0-1.3+deb11u3
  Version table:
 *** 7.74.0-1.3+deb11u3 100
        100 /var/lib/dpkg/status
libcurl3-gnutls:
  Installed: 7.74.0-1.3+deb11u3
  Candidate: 7.74.0-1.3+deb11u3
  Version table:
 *** 7.74.0-1.3+deb11u3 100
        100 /var/lib/dpkg/status

# uname -a
Linux shipxplorer 5.4.0-1073-raspi #84-Ubuntu SMP PREEMPT Sat Oct 15 11:13:36 UTC 2022 aarch64 GNU/Linux

Also -- I switched from building my own AIS-catcher to pulling in yours into the Docker Build with:

FROM ghcr.io/jvde-github/ais-catcher:edge AS build
...
COPY --from=build /usr/local/bin/AIS-catcher /usr/local/bin/AIS-catcher

Either way (building myself or using your build image) makes no difference in behavior.

dziban303 commented 1 year ago

I've been running v0.39 without trouble on my Pi3, happily feeding aprs.fi. To help @kx1t troubleshoot I ran it on a Pi4 4GB with a similar environment, and also ran into the out of memory error.

$ AIS-catcher -q -v 10 -d 136 -s 1536K -gr rtlagc on biastee on tuner 42 -u 192.168.0.180 10110 -H http://aprs.fi/jsonais/post/B-------p ID ????? PROTOCOL aprs INTERVAL 30
AIS-catcher (build Nov  3 2022) v0.39
(C) Copyright 2021-2022 jvde-github and other contributors
This is free software; see the source for copying conditions.There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Device selected: Realtek, RTL2838UHIDIR, SN: 136
Found Rafael Micro R820T/2 tuner
Device: RTL2838UHIDIR
Settings: rate 1536K tuner 42.000000 rtlagc ON biastee ON
HTTP: start server (http://aprs.fi/jsonais/post/B-------p).
Allocating 15 zero-copy buffers
[AIS engine v0.39]                      received: 29 msgs, total: 29 msgs, rate: 2.82748 msg/s
^C----------------------
[AIS engine v0.39]                      total: 32 msgs
HTTP: error - Out of memory (http://aprs.fi/jsonais/post/B-------p)
HTTP: server response -
HTTP: stop server (http://aprs.fi/jsonais/post/B-------p).

(edited for privacy)

The Pi4 is running Raspberry OS bullseye: Linux pi4 5.15.74-v8+ #1595 SMP PREEMPT Wed Oct 26 11:07:24 BST 2022 aarch64 GNU/Linux. Curl-related packages installed on the Pi4:

curl/stable,now 7.74.0-1.3+deb11u3 arm64 [installed]
libcurl3-gnutls/stable,now 7.74.0-1.3+deb11u3 arm64 [installed,automatic]
libcurl4-openssl-dev/stable,now 7.74.0-1.3+deb11u3 arm64 [installed]
libcurl4/stable,now 7.74.0-1.3+deb11u3 arm64 [installed,automatic]
python3-pycurl/stable,now 7.43.0.6-5 arm64 [installed,automatic]

Meanwhile the Pi3 is on raspberry OS buster: Linux raspberrypi 5.10.103-v7+ #1529 SMP Tue Mar 8 12:21:37 GMT 2022 armv7l GNU/Linux, and the Pi3's curl-adjacent packages installed are:

curl/oldstable,now 7.64.0-4+deb10u3 armhf [installed]
libcurl3-gnutls/oldstable,now 7.64.0-4+deb10u3 armhf [installed,automatic]
libcurl4-openssl-dev/oldstable,now 7.64.0-4+deb10u3 armhf [installed]
libcurl4/oldstable,now 7.64.0-4+deb10u3 armhf [installed,automatic]
python3-pycurl/oldstable,now 7.43.0.2-0.1 armhf [installed,automatic]

Note I'm running AIS-catcher outside of Docker in both cases.

jvde-github commented 1 year ago

Thanks! Will have a look tonight to see if I can reproduce.

jvde-github commented 1 year ago

The hints were helpful. I did not see a difference with the version I installed except that in dziban's post seems to suggest it was a 64 bit version (and I had 32 bit running). So installed that as well on my RPi 4 and managed to get the same error message for APRS only.

With trial and error the culprit seems to be sending the content length in the form definition. As the message is a nul-terminated string (text) I don't think that is needed so removed it. For me it works now on 64 bit. It seems that you need to explicitly cast the length to a long (which is another way to solve this).

Can you try again and see whether this has fixed the issue?

Thanks, Jasper

kx1t commented 1 year ago

Hey Jasper -- thanks for the quick help. I am rebuilding my container and I will let you know. (I'll update this comment with the results to avoid spamming)

UPDATE -- Appears to be working. HTTP posting appears to ignore the -q option as it does write to the screen every INTERVAL seconds, but I'd think that's a minor issue (and it helps in debugging).

Command line:

/usr/local/bin/AIS-catcher -u 127.0.0.1 34995 -u 5.9.207.224 xxxx u ais.vesselfinder.com xxxx
 -q -v 60 -d AIS -gr tuner auto rtlagc ON -H http://aprs.fi/jsonais/post/zxxxxxxxV ID xxxx PROTOCOL aprs INTERVAL 30

Results in logs:

HTTP: server response - {"protocol":"jsonais","command":"post","result":"ok","description":"received jsonais POST successfully"}

Thanks again for your speedy replies! Feel free to close the issue

jvde-github commented 1 year ago

Great! Thanks for raising and dziban for his additional insight. Will close.

ps q switch only controls nmea screen output to avoid overflow of nmea lines in logs. Should not have an impact on udp and http output. For http you can give switch response off