Readsb isssue with --net-json-port=... or --net-connector=...,json_out

[sign0]

Hi,

I would like to get the JSON output via a TCP stream with --net-connector=...,json_out or with --net-json-port=... (with --net-only mode)

I use the #dev branch of your readsb fork, but despite the presence of the --write-json-globe-index option (with or without --write-json-every=1 and/or --write-json=... and/or --write-globe-history=...), I had no output during these tests.

Do you have an idea of the problem ? Perhaps I missed an important consideration ?

By the way, great work for this very useful readsb fork and the Tar1090 stack ! ;)

[wiedehopf]

Test longer.

[wiedehopf]

Also post your exact command line.

pgrep -a readsb

[sign0]

Sorry but I haven't had time to look into this problem again until today ...

The result of pgrep -a readsb :

340 /usr/local/bin/readsb --quiet --net-only --net-heartbeat 60 --net-sbs-port=30003 --write-json-every=1 --write-json-globe-index --net-json-port=1234 --lat=XX.XXXXXXX --lon=X.XXXXXXX --net-connector=XXX,33002,sbs_in

I left running for several minutes, and no output on port 1234 (JSON output). Ideally I would like to have a JSON output as a stream every each seconds...

On the other hand, the output on port 30003 is ok (it shows me the SBS stream which is in input)

[sign0]

There is an error message related to awk on startup... Maybe it is related ?

Here is my startup log :

tar1090 database version: be82d07
tar1090 version: f6c6519
[cont-init.d] 02-tar1090-update: exited 0.
[cont-init.d] 03-tar1090-copy: executing...
[cont-init.d] 03-tar1090-copy: exited 0.
[cont-init.d] 04-tar1090-configure: executing...
[cont-init.d] 04-tar1090-configure: exited 0.
[cont-init.d] 05-tar1090-gzip: executing...
[cont-init.d] 05-tar1090-gzip: exited 0.
[cont-init.d] 06-adsbexchange-stats: executing...
[cont-init.d] 06-adsbexchange-stats: exited 0.
[cont-init.d] done.
[services.d] starting services
awk: option `-W interactive' unrecognized, ignored
awk: option `-W Interactive' unrecognized, ignored
[readsb] Mon Jun 15 11:01:48 2020 CEST  wiedehopf dev starting up.
[readsb] Version: f69002e Sun Jun 14 15:22:24 2020 0200: better handle type for VRS
[readsb] struct sizes: 1728, 24, 64
[readsb] Net-only mode, no SDR device or file open.
[services.d] done.
[readsb] Basestation TCP input: Connection established: XXX (XXX) port 33002
awk: option `-W interactive' unrecognized, ignored

Edit : I replaced my gawk version with default mawk, I no longer have an error message, but that doesn't change anything for the option --net-json-port which doesn't work ...

[wiedehopf]

You're not giving it the correct argument.

 # ./readsb --help
      --write-json=<dir>     Periodically write json output to <dir>

[wiedehopf]

Oh nevermind you want the json on the TCP ... or do you? I'm really not sure what you want.

[sign0]

I also tested with all these arguments, but it doesn't work better... :(

339 /usr/local/bin/readsb --quiet --net-only --net-heartbeat=1 --net-sbs-port=30003 --write-json-every=1 --write-json=/run/readsb --write-json-globe-index --write-globe-history=/var/globe_history --globe-history-heatmap=1 --net-json-port=1234 --net-connector=XXX,33002,sbs_in

[sign0]

I would like the aircraft.json (or json tiles) to be push via TCP, to avoid reading / locking the file every x seconds on the hard drive ...

[wiedehopf]

./readsb --net-only --net --net-connector localhost,30005,beast_in --quiet --net-json-port 1234 --write-json-globe-index```

So this is working and producing output on 1234.
But it's not what you want and i don't have what you want.

Use the standard approach and do it with a webserver, i don't fancy building in another TCP output.

[wiedehopf]

The --net-json-port is not what you want .... the help describes as well that it's different. TCP json position output

So it puts out new positions for aircraft. As such you will only get output if you have aircraft for it.

What is the issue with a /run directory using tmpfs and using nginx or w/e webserver to get the file from there?

[wiedehopf]

I think your main issue is using some ancient linux with init.d.

Just use ubuntu or debian, then you can install the debian package and the aircraft.json will be in /run/readsb in a tmpfs ....

[wiedehopf]

Anyhow .... do the following as root:

mkdir /mytmpfs
mount -t tmpfs -o size=512m tmpfs /mytmpfs
./readsb --net-only --net --net-connector localhost,30005,beast_in --quiet --write-json /mytmpfs

That gives you aircraft.json in a tmpfs.

[sign0]

If the --net-json-port push only the "new" positions, it's even better than I would hope ! It is much more elegant than parsing a full JSON or some planes have not been refreshed ... :)

In fact what interests me is to centralize several streams (BEAST or RAW or SBS) via Readsb and have a "generic" JSON output which contains both "metas data" (Wind, ...) and the MLAT array (for BEAST/RAW feeds).

I have containerized your work, everything runs via the docker image debian:stable. I will try to dig into what you told me : tmpfs, latest debian, ...

I therefore deduce that the --net-json-port works at least in the case of BEAST given your test. Is this the case for all other input formats ?

I will keep you informed of my progress ... Thank you for your help ! :)

[wiedehopf]

Should work with all data inputs as long as they have positions for planes. No new position, no output.

The frequency of position updates depends on speed / altitude / track changes. You can change the max timeout before it will output a new position for each plane when it's flying straight same speed same altitude.

  --json-trace-interval=<seconds>
                             Interval after which a new position will
                             guaranteed to be written to the trace and the json
                             position output (default: 30)

[wiedehopf]

Maybe it wasn't really a basestation stream you were putting in?

This is what SBS input needs to look like

MSG,3,1,1,3C5EE3,1,2020/06/15,12:30:22.948,2020/06/15,12:30:22.948,,39000,,,50.98057,9.19853,,,0,,0,0

apt install nc
nc XXX 33002

Check what your basestation looks like maybe it's a different format.

[sign0]

I'm sure it's SBS... (from ADSBHub) A small extract from the output with netcat :

MSG,3,0,0,86CF91,0,2020/06/15,10:33:13.000,2020/06/15,10:33:13.000,,14575,,,35.118896,139.880783,,,,,,
MSG,4,0,0,86CF91,0,2020/06/15,10:33:13.000,2020/06/15,10:33:13.000,,,348,32,,,-448,,,,,
MSG,1,0,0,86D9B8,0,2020/06/15,10:33:13.000,2020/06/15,10:33:13.000,JAL86,,,,,,,,,,,
MSG,3,0,0,86D9B8,0,2020/06/15,10:33:13.000,2020/06/15,10:33:13.000,,1300,,,35.603989,139.829178,,,,,,
MSG,4,0,0,86D9B8,0,2020/06/15,10:33:13.000,2020/06/15,10:33:13.000,,,143,269,,,-768,,,,,
MSG,1,0,0,86DDB2,0,2020/06/15,10:33:13.000,2020/06/15,10:33:13.000,JAL331,,,,,,,,,,,
MSG,3,0,0,86DDB2,0,2020/06/15,10:33:13.000,2020/06/15,10:33:13.000,,39975,,,35.22794,136.517883,,,,,,
MSG,4,0,0,86DDB2,0,2020/06/15,10:33:13.000,2020/06/15,10:33:13.000,,,381,258,,,64,,,,,
MSG,3,0,0,86E0BE,0,2020/06/15,10:33:13.000,2020/06/15,10:33:12.000,,25050,,,39.301819,139.727905,,,,,,
MSG,4,0,0,86E0BE,0,2020/06/15,10:33:13.000,2020/06/15,10:33:12.000,,,382,25,,,64,,,,,
MSG,1,0,0,86E79A,0,2020/06/15,10:33:13.000,2020/06/15,10:33:13.000,ANA8571,,,,,,,,,,,
MSG,3,0,0,86E79A,0,2020/06/15,10:33:13.000,2020/06/15,10:33:13.000,,40250,,,37.450298,140.018005,,,,,,
MSG,4,0,0,86E79A,0,2020/06/15,10:33:13.000,2020/06/15,10:33:13.000,,,539,15,,,-384,,,,,

It's just to test the ideal configuration, after I plan to mainly aggregate BEAST and RAW (to have wind, mlat, etc) ...

[sign0]

Json output work with beast inputs ! :p I'm almost there, I'm just missing the problem of SBS input.

I notice that if I put SBS as input, listening to the output in RAW sends me empty messages ... *0000; or 1 for the BEAST. However if I listen to the SBS output, I have the same thing as input.

For SBS, I use the ADSBHub stream. I can send you by email the address of my proxy so you can test on your side...

[wiedehopf]

That was an oversight on my part, please git pull and recompile.

[sign0]

It works wonderfully, thank you very much !! :D

I have a suggestion for a new feature/improvement of readsb for my use case and probably also for tar1090 (and ADSBExchange). I open a new ticket for that or I make my suggestion here ?

[wiedehopf]

You are really gonna make me answer another time just to tell you where to ask for me to change the code for you? ........

Anyhow glad it's working.

[sign0]

This is why I proposed a new ticket because I am well aware that a suggestion is not the same degree as a bug ! ;)

My idea was to add a 4th argument at the end of --net-connector to specify an id to the stream, and retrieve the ids that participated in the update of a position/aircraft in a JSON array (like ModeSMixer2) . I will try to do a PR, but I haven't done C since my studies, it could quickly turn short...

Anyhow glad it's working.

[wiedehopf]

You'll probably spot there is already a receiver id, but it's not used for much except have a reference for surface CPR.

Also net-connector already has 4 arguments ;P And currently it can't have non-empty arguments due to the use of strtok.

Anyhow ..... i'll have a look.

[aqeel-afzal]

It works wonderfully, thank you very much !! :D

I have a suggestion for a new feature/improvement of readsb for my use case and probably also for tar1090 (and ADSBExchange). I open a new ticket for that or I make my suggestion here ?

@sign0

Have you managed to make it work with SBS input only?

[wiedehopf]

@aqeel-afzal that's rather unspecific. just say what you're trying to do.

[aqeel-afzal]

I am feeding SBS stream to readsb instance. SBS out is working but JSON out is not showing anything

readsb --net --net-only --net-ingest --net-sbs-in-port 30104 --net-sbs-port 30003 --net-bi-port 30004 --net-json-port 30005 --write-json /run/readsb --json-trace-interval 1

nc localhost 30005 doesn't show anything

[wiedehopf]

30005 is usually beast output, so this is very confusing.

Are you sure that's the command line that is running? Post the output for this:

pgrep -a readsb

[wiedehopf]

Hmmm ... let me check. It seems the json output indeed doesn't work with sbs input ... i'll check it.

[wiedehopf]

Should be fixed.

I'd highly recommend leaving beast out at 30005 if you ever want to feed anyone data as they usually expect it on that port.

You can just add the json port on another port :)

[aqeel-afzal]

I was just trying different port for testing. I'll use different port for json :)

Thank you and really appreciate your response.

[wiedehopf]

May i ask, what's your data source as SBS?

[aqeel-afzal]

I am getting SBS data from ADSBHub