search

wiedehopf / graphs1090

Graphs for readsb / dump1090-fa / dump1090 (based on dump1090-tools by mutability)
MIT License
407 stars 42 forks source link

automation for netcat to get data from additional server e.g. flightfeeder #3

Closed Zp9Zp9 closed 5 years ago

Zp9Zp9 commented 5 years ago

Hello, i currently use command nc -d 192.168.2.122 30005 | nc 127.0.0.1 30004 & for piping the traffic from a loaned flightfeeder into my other pi, where running fr24, ps, rs24 and all the other adsb services with use of dump1090.

Problem is, if connection drops, nc cannot rebuild it - and call nc when the connection is etablished opens a second (third, forth,...) connection - and they all transmits data. So flights will be reportet twice or many at same time if this occurs. I cannot add the nc command in cron except the single one at reboot.

Question: Is there a way to detect loosing the communication (remote traffic = 0) AND give console command "nc" (one time!) with parameters as shown in this case ?

Can anybody implement this, please? It seems to be a good way to get the Flightfeeder data for all the other services, too... the reception of the Beast with the jetvision antenna is incredible ;-)

Regards Mirco

wiedehopf commented 5 years ago

Check out these scripts: https://github.com/wiedehopf/combine1090

Zp9Zp9 commented 5 years ago

I would not install another dump, so i need only the injection routine... which is getting traffic data from remote port 30005 and send it to local 30004. Im not very firm in programming oder scripting, so please tell me, which part i can modify...

(you can also write in german, if possible)

Regards, Mirco

wiedehopf commented 5 years ago

I see, this is socat with a 90 second timeout on inactivity. (no data) You will have to use it in a while loop with some sleep.

while sleep 5
do
  socat -T 90 -d -u TCP:192.168.2.122:30005 TCP:127.0.0.1:30004
done

This what i use in the script. What dump1090 do you use? Maybe i should split up the redirection bit and the display so it's not depending on dump1090-fa.

Zp9Zp9 commented 5 years ago

Maybe just give possibility to setup the redirection (or other function calls depending on measuring stats e.g. temperature or something) in the config of graph1090 - or give small script with install routine and config file...

Actual i use piaware on Raspi 3b+ with the feeder addons for FR24, OSN, PF, RadarBox, ADSBexchange and ADSBHub. An additional loaned FlightFeeder with its own Antenna is located 5km northwest at EDVK Airfield (at the moment for testing at same position as my own pi with RTLSDR).

I will give a further Raspi for feeding the FF data from EDVK - included SSH, DynDNS etc.

Problem to solve was only to get the data out of the FF without longer breaks ...

wiedehopf commented 5 years ago

No no, i won't change graphs1090. I meant splitting out the redirection in combine1090 from the second dump1090-fa instance.

If you already use piaware with feeder addons just install combine1090. The extra dump1090-fa running in net-only mode won't take any resources.

Then change the target port and you'll have what you want.

Zp9Zp9 commented 5 years ago

there is running a dump1090-fa, for what is the additional ? i think if i inject into 30004 the present dump1909-fa will recieve and process the data. MLAT injecting 30105 i took out of config due to ff regulations not to use mlat data für other feeders.

wiedehopf commented 5 years ago

Normally the additional dump1090-fa is for aggregating multiple feeds.

But if you don't feed it data it's just gonna sit around and do nothing. Which is not a problem. (you could disable it: sudo systemctl disable combine1090-dump with it disabled it won't start on next reboot.)

dump1090-fa won't forward the MLAT data to the feeders because the data is marked as MLAT. (unless you change the dump1090-fa configuration to explicitely forward mlat)

Zp9Zp9 commented 5 years ago

they should not forward the remote MLAT... remote device is an loaned flightfeeder and the fa regulations allowing NOT using mlat data für other feeders than fa...

My only problem is to etablish stable data fetching from the flightfeeder to my own Raspi for injecting in Port 30004 to the existing dump1090-fa, which also processes the local rtlsdr data to all the feeders

wiedehopf commented 5 years ago

So install the combine1090 script already. Disable port 30105 if you want. Change the TARGET port.

Ignore the additional dump1090-fa running.

What is the problem?

Zp9Zp9 commented 5 years ago

its not working... the both instances seems to bite each other, no data will transferred to the service servers. (of course, 122 flight feeder still sends to flightaware, but the 118 does nothing) `

Append all IP addresses running dump1090/piaware in the next line, 127.0.0.1 is just an example you should replace it

SOURCES= 192.168.2.122

which ports to collect beast data from (for piaware/dump1090-fa this is 30005 for ADS-B and 30105 for MLAT)

PORTS= 30005

for the IP addresses in SOURCES all PORTS are used, if you want to only forward a single or non-standard port use the following:

uncomment and add IP addresses with ports for redirection

CUSTOM=127.0.0.1:29005 127.0.0.1:29999

where the data shall go, for display under /combine1090 leave it as is.

TARGET=127.0.0.1:30004

options for dump1090-fa where all the data goes

RECEIVER_OPTIONS="--net-only"

DECODER_OPTIONS="--max-range 360"

NET_OPTIONS="--net --net-heartbeat 60 --net-ro-size 1000 --net-ro-interval 0.2 --net-ri-port 29001 --net-ro-port 0 --net-sbs-port 0 $ JSON_OPTIONS="--json-location-accuracy 1"

`

I stopped it now alt previous function is given: My own Raspi (118) with RTLSDR, piaware (incl dump1090-fa), added feeders for the other services. Loaned FlightFeeder (122) without any changes

Getting Data with using nc -d 192.168.2.122 30005 | nc 127.0.0.1 30004 & on the 118 device. Works fine, but sometimes data link breaks down and i have to restart it with the nc command.

And i will only automatize this data link without major changes to the existing and running setup.

wiedehopf commented 5 years ago

I gave you two possible solutions, if you are unable to use them that's unfortunate.

After changing the combine1090 configuration you need to restart it, it's explained on the github page for the project. The flightaware forum is a much better place than github to look for a solution to such problems.

Zp9Zp9 commented 5 years ago

of course i have restarted... problem is, in the moment your script is working, the "original" dump1090-fa seems to get no further data (maybe overwritten config/redirected to the new dump1090-fa) and so its not feeding any data.

i will check the solution with socat but i have some trouble where to put the commands... as i said im not really fit in programming and scripting,...

wiedehopf commented 5 years ago

The second dump1090-fa does not interfere with the main dump1090-fa instance.

I see the problem: you commented these two lines.

#RECEIVER_OPTIONS="--net-only"
#DECODER_OPTIONS="--max-range 360"

that means the second dump1090-fa will try and use the dongle. Why would you do that? Just leave the second dump1090-fa options as they are.

Zp9Zp9 commented 5 years ago

with outcommented lines its also dont work, the first dumper sends no data, neither from local nor from remote...

i want only to inject the remote data into the first dump1090-fa... he ist listening at 30004... i need only the "watchdog" for the data link...

wiedehopf commented 5 years ago

reboot the pi and it will work.

the second dump1090-fa won't interfere if it's running with the option net-only. Let's check what the second dump1090-fa is doing

pgrep -fa dump1090-fa
Zp9Zp9 commented 5 years ago

The command gives

pi@raspi:~ $ pgrep -fa dump1090-fa 520 /usr/bin/dump1090-fa --net-bo-port 30005 --fix --device-index 0 --gain -10 --ppm 0 --max-range 360 --net --net-heartbeat 60 --net-ro-size 1000 --net-ro-interval 1 --net-http-port 0 --net-ri-port 0 --net-ro-port 30002 --net-sbs-port 30003 --net-bi-port 30004,30104 --json-location-accuracy 2 --lat 51.36538 --lon 9.43083 --write-json /run/dump1090-fa --quiet

wiedehopf commented 5 years ago

The second dump1090-fa isn't even running, so how would it interfere?

Seems to be working just fine. Which map are you looking at where there are no planes?

Zp9Zp9 commented 5 years ago

at the moment as i said i uninstalled the combine1090 script and manually started data collecting with nc as stated in start post. so in the moment no second dump is running.

if i reinstall combine1090, editing config, restarting the service and even when i restart the whole pi, the skyview of combine shows the remote flights, the original at port 8080 shows the local flights. a few seconds later the remote disappear, at no time one of the screens show remote and local together (as he does with the actual "nc" solution). Data bandwidth and status at the statistics on feeder server said im not transmitting data, graph1090 also shows no data

wiedehopf commented 5 years ago

Did you also change this line?

NET_OPTIONS="--net --net-heartbeat 60 --net-ro-size 1000 --net-ro-interval 0.2 --net-ri-port 29001 --net-ro-port 0 --net-sbs-port 0 $

Only change the SOURCE, TARGET and PORTS. Don't change anything else. Best to remove /etc/default/combine1090 and then install again so you have the original version.

You basically changed the second dump1090-fa from combine1090 to occupy the same ports as the main one. Because when you remove the option --net-bo-port 0 for example, it reverts to the default of 30005.

I never asked you to change that stuff. Even told you a command to disable the second dump1090-fa completely without affecting the data redirection. Figure it out yourself please i'm done.

Zp9Zp9 commented 5 years ago

I can only say it again: Its not running, its not installed this moment... all of greped info is from first dump1090-fa... i will try it again, of course, but why i must buy a bus instead a ticket to travel?

I only want a convenient way to establish and keeping stable the datalink between the port 30005 of the flightfeeder and port 30004 of the dump1090-fa on my own Raspi...

I asked you, because your script shows that you found a way to solve this. This doesn´t mean, that i will dice all in another because the system is stable except the datalink issue.

Of course i can use a windows machine with VRS to make a stable data frowarding connection - but the glance of Raspi is the standalone functionality... there must be a quick small solution and my hope was that you can help me to find this even i had no knowledge about programming, scripting or autostarting a do...while-loop...

Zp9Zp9 commented 5 years ago

for this first idea was having a "trigger" in graph1090, to give the nc command if remote connections count falls to zero...

wiedehopf commented 5 years ago

I won't make the bus "just a ticket".

You can make the bus "just a ticket" by disabling the second dump1090 service via sudo systemctl disable combine1090-dump and rebooting. (which is not necessary unless you modify it's parameters and shoot yourself in the foot)

I've found a way for it to work and make it convenient to use. And if you just change the ports the bus will work just fine. I can find a lot of other ways to make it work but most people are even confused by the concept of ports. So a turnkey solution to aggregate multiple feeds for viewing.

If you don't know where to put a the socat command then the only answer i will give you is: you have to make the bus work. Probably by now i should have just forked the combine1090 project and taken out the second dump1090-fa as it would've taken less time than typing all this.

wiedehopf commented 5 years ago

So i've made an option to install the redirection portion of combine1090 only: https://github.com/wiedehopf/combine1090#install-data-redirection-only

Probably not a bad idea, please excuse my previous impatience.