Public syslog server + signal tracking web page / API

[madpsy]

Heads up that I've created a public syslog server which can ingest logs from an iGate. It lets the user view their own logs as well as plots data on maps - including signal strength etc. This is purely using the syslog output messages (does not have anything to do with aprs.fi !).

There's a rudimentary API and GeoJSON endpoint in case that's useful.

The website is: https://lora.link9.net/

Maybe you want to set 'lora.link9.net' port '1514' as the default for syslog settings (or the placeholder text?) - I don't mind if it was even enabled by default but that's up to you!

Any questions /concerns / comments please ask!

[IU5HKU]

Thank you so much, i'll give it a try

[IU5HKU]

There is something strange in the reports, i'm used to get long distance contacts, even more than the ones i see in your map, but this time these contacts are not mine, they aren't in the list of "Stations heard directly by" than you can read in aprs.fi website. Your site assign to me some packets digipeated by IQ5VK-11, that is the local club igate-digipeater. Ok, maybe i've igated some packets first heard by IQ5VK-11, but they can't be stated as i've heard directly it. If you want you can have a look at my reports for F5ZVD-4 , F1ZLL-4, 3A2ARM-10. Another strange thing is that in the "Stations who heard directly on radio" list of these callsign, you can't find IU5HKU-10 nor IQ5VK-11. I don't understand this behaviour. For the moment i disconnect my igate from your site, i don't wanna pollute the db with "strange" data, let me know. Anyway, i like the whole things, i find your service very useful.

[madpsy]

Thanks for testing it out. I knew there would be a few bugs and you found one!

Should be fixed now if you wouldn't mind enabling it again?

[IU5HKU]

Sure, the syslog is just enabled now, i'll keep it under control in the next hours/days, thank you for this great service :-)

[madpsy]

I see your traffic again - shall keep an eye on it.

Of course let me know if you are seeing too few or too many than you expect. I removed the problematic ones and hopefully my logic is right.

It will now ignore Digipeated packets as that isn't really the point of the service (maps etc) so I think this is sensible :-)

[richonguzman]

this is great for testing !!! nice!

[richonguzman]

maybe this could be created as a Repository link with info and such and I could link to it in the readme and wiki ?

[Mane76]

There is something strange in the reports, i'm used to get long distance contacts, even more than the ones i see in your map, but this time these contacts are not mine, they aren't in the list of "Stations heard directly by" than you can read in aprs.fi website. Your site assign to me some packets digipeated by IQ5VK-11, that is the local club igate-digipeater. Ok, maybe i've igated some packets first heard by IQ5VK-11, but they can't be stated as i've heard directly it. If you want you can have a look at my reports for F5ZVD-4 , F1ZLL-4, 3A2ARM-10. Another strange thing is that in the "Stations who heard directly on radio" list of these callsign, you can't find IU5HKU-10 nor IQ5VK-11. I don't understand this behaviour. For the moment i disconnect my igate from your site, i don't wanna pollute the db with "strange" data, let me know. Anyway, i like the whole things, i find your service very useful.

Please consider the way APRS-IS is working - if one iGates receives a station it sends this info to APRS-IS and this is then displayed on the web - if multible iGates receive a station a filtering starts at the APRS-IS servers and only one (!) receiving iGate is displayed on the web. Usually the "fastest" one is displayed. Now with syslog you see all (!) iGates which have received a single frame.

So should be pretty ok.

[IU5HKU]

Yes, for sure, but after the fix madpsy did, seems that infos are more suitable to the "real" situation, you can't consider a second-hop packet as directly heard by you...but now it works ok, stats are correct. Well done.

[S57PNX]

Can you add a button to collapse the left side of the screen (live logs) and make the map full size?

[madpsy]

@IU5HKU Glad the fix worked - you are still top for distance :-)

@richonguzman It doesn't have a public code repo but there's an open API (https://lora.link9.net/api/) and a help page.

I was thinking you could make it the default host/port values for syslog settings (but off by default) and maybe a link to the site for convenience. Or... just link to the site in the readme/wiki and people can manually configure it if they want. Whatever you think is best for how you prefer to do things - I appreciate we haven't spoken before!

[madpsy]

@S57PNX should be there now if you refresh

[S57PNX]

Thanks, looks very nice - but I can see the button in Chrome while I don't see it in Safari?!

I've added one of my gates (S58W-13) to your syslog to test, but I find some unexpected results - as if somehow digipeated trackers show as directly heard by my gate.

E.g. S54Z-11 shows as being received by S58W-13, but I am sure it was digipeated by S55YLI-10. All these suspect cases show

while the real direct contacts show this:

[madpsy]

Do your spots look better now?

Safari is a funny animal. Does it show if you maximise the window?

[S57PNX]

Yes, my spots look correct now, thank you. And the button has appeared in Safari too!

[S57PNX]

One more nitpick: in the pop-up bubble, the Heard: timestamp is off by one hour for me. A station that was just heard for the first time shows as Heard: 1h 0m. I am in GMT+1 observing DST, so effectively GMT+2.

Can you add "ago" at the end of 1h 0m ?

[madpsy]

I knew that would happen - initially my intent for this was just for a UK club so didn't take other TZ's into account. Now I'll have to rethink a few things. Timezones are always an 'interesting' problem.

Have added 'ago' again (was there before but removed it - will see how it looks)

[madpsy]

Quick update. I've added an unauthenticated MQTT feed which might be useful to some folks.

Demo: https://lora.link9.net/mqtt-demo.html

[IU5HKU]

Hi madspy, there is something strange again in parsing the syslog, look:

HB4LO-9>APLRT1,IQ5VK-4*:!/6tx9P4UXO`GQHB4LO tracker Bat=4.05V (89%)

this is a packet digipeated by IQ5VK-4 and heard by my igate, IU5HKU-10. Your stats assign the contact with HB4LO-9 to my igate. Strange enough i can't find HB4LO-9 into IQ5VK-4 list of "directly heard stations" on aprs.fi, i can't say nothing about IQ5VK-4 syslog, i haven't access to it.

[madpsy]

Seems I committed a change and it was behind the main branch somehow so added my changes but reverted my previous commit to fix that!

Should be fixed now, for the second time

[richonguzman]

(I'm reading all of this, will return from Country Holidays on Monday to be online :P )

[S57PNX]

Another funny gltch - S58K-10 is plotted in a completely wrong spot, maybe 80km from its true location.

Correct plot from aprs.fi:

Wrong plot on lora.link9.net:

[madpsy]

@S57PNX If I look now it seems correct, right? Looking back in the logs its locations hasn't changed at all so how it managed to plot there is a mystery just now!

Let me know if you see it again though as that's a strange one.

[madpsy]

I think it was a bug with the convertCoordinates() function which was needed previously but not now. You may need to hard refresh the page to get the new JavaScript code.

If you can still make it break please use the 'Link' button at the top and paste the full link in here for me, thanks!

[S57PNX]

@madpsy thank you, it is looking correct now. I don't know how it happened to be, I just noticed it one day :)

Out of curiousity, what kind of resources do you have to run this database/web site? Will it handle an influx of new users?

[madpsy]

Just now it's running on a small VM (1 CPU / 4GB RAM) but is under no load:

load average: 0.05, 0.01, 0.01

If it becomes a problem can just throw more resource at it easily enough :slightly_smiling_face: Just now the most resource intensive page is the 'Top 10' page. If that starts becoming slow I can add some caching in various places so not overly concerned.

The MQTT page (now at http://lora-aprs.live) doesn't create any load as the processing is done on the client side. That is very much just a 'demo' for now as has a few known issues.

[madpsy]

Have added SNR / distance / coverage heatmap for iGate owners, it's under the 'Coverage' link on the main page.

Example: https://lora.link9.net/coverage.html?rx_callsign=MB7UWI-10&time_period=48

[iw2ejh]

would it be possible to have the possibility to configure two syslog servers in the configuration, so we can have a shared public one, and a personal local one?

[jclifford73]

would it be possible to have the possibility to configure two syslog servers in the configuration, so we can have a shared public one, and a personal local one?

I would also love to have configuration for 2 syslog servers.

73 John GW4BVE

[richonguzman]

I guess I can add the second syslog posibility

but dont you think this would go on slowing more and more igate process?

can one syslog be linked into another?

[iw2ejh]

@richonguzman 1-)but dont you think this would go on slowing more and more igate process?( -- I honesty do not know what is the impact of sending log to syslog. You may valuate that based on the various task. If this took time and resources, obviously this function can be omitted. Better give priority to the original tasks.

2-)can one syslog be linked into another?( -- I couldn't find an easy way to do that, some software are available but need installation and the data forwarding is not their primary function. Standard Syslog servers usually do not have that capability.

[richonguzman]

2-)can one syslog be linked into another?( -- I couldn't find an easy way to do that, some software are available but need installation and the data forwarding is not their primary function. Standard Syslog servers usually do not have that capability.

usually syslog servers are more powerful than a ESP32 board this is why I asked ;)

[iw2ejh]

usually syslog servers are more powerful than a ESP32 board this is why I asked ;)

If you have a dedicated sw running on a Server, yes, but at home we usually use a light app running on a NAS (in my case) that do not have any features other than recording data.

[madpsy]

Just FYI if you're sending syslogs to lora.link9.net there's a 'firehose' MQTT (JSON format) feed to read them back out again, example:

mosquitto_sub -h hydros.link9.net -p 8883 --capath /etc/ssl/certs/ -t lora_aprs/MB7UNH-10/logs | jq

That will show all logs for iGate MB7UNH-10. You can do all the usual MQTT filtering using + and # etc and the structure should be clear from the page below. I wrote a demo of a pure JS implementation using leaflet which only uses this feed to create a live map.

More info here: https://lora-aprs.live/mqtt/

[S57PNX]

The problem with one syslog server forwarding to another server, or alternatively having a piece of software to split traffic to two syslog servers, is introducing a single point of failure. If this server or splitter fails, you lose both syslogs.

This is why I would prefer to split at the source on ESP32, and have a possiblity to send to two syslogs. It's somewhere on my todo list, but won't have any time to work on my todo for multiple weeks.

[madpsy]

One more nitpick: in the pop-up bubble, the Heard: timestamp is off by one hour for me. A station that was just heard for the first time shows as Heard: 1h 0m. I am in GMT+1 observing DST, so effectively GMT+2.

@S57PNX Date/times in general should now be fixed throughout

[S57PNX]

Yes, time works ok now, tnx. And I like the new improvements (Moving etc.) However, one of my use cases for your web page is to track the perfarmance/behaviour/health of my gate(s), so I am curious which stations it is picking up. Can you add a filter to the map view to display only stations heard by a single gate? Like you can select the gate in the Coverage view.

Currently, a tracker will "jump" between two gates that heard it, depending on which gate heard it last. I would rather see that "my" gate heard it 10 minutes ago, with an associated path and SNR etc. displayed, than some other gate heard it just now.

[madpsy]

Does this view not do that?

https://lora-aprs.live/igate.html?rx_callsign=S58W-13&time_period=20

or this view?

https://lora-aprs.live/history.html?rx_callsign=S58W-13&tx_callsign=S57PNX-14

or do you mean something else?

[S57PNX]

Oh yes, the first view ( https://lora-aprs.live/igate.html?rx_callsign=S58W-13&time_period=20 ) does exactly what I wanted, thanks. I just didn't realize that I needed to click on S58W-13 in the logs or on the maps to get this info. :)

[madpsy]

Good stuff! Yes, frontend design (UX etc) is not my strong point, as you can tell :-)

[pdevlin13]

Thanks for this madspy, it is a really useful tool. I thought I had a problem, but realised it was a beacon not showing on APRS but was in the log as Direct RX and so deleted that comment :-) Thanks Paul MW0PDV

[madpsy]

Glad to hear - if you do notice anything wrong obviously let me know though - it's quite a complicated beast now with many opportunities for bugs!

[madpsy]

Have made a 'beta' frontend page for tracking elevation vs SNR vs iGate distance. Uses the public 'track' API endpoint so all of this is purely frontend code you can see via 'view source' and feel free to improve! Unsure if it's interesting but I thought it was kind of cool to see. Change tx_callsign to whatever you want. Example:

https://lora-aprs.live/elevation.html?tx_callsign=WA4OSH-1

'points' can be changed via the +/- buttons

[S57PNX]

@madpsy I noticed that you added Countries to your page - I assume that you mean the country of the tracker (TX) location? It seems that you define the country by coordinates, not by callsign.

S59HIJ-7 is located in Slovenia, not Croatia. Whereas S52ZO-13 indeeds sits in Croatia :)

[S57PNX]

And when it comes to iGate Unique trackers, S58W-13 is the only gate listed for Croatia, even though it is not located in Croatia.

[madpsy]

@S57PNX Yeah it's based on the origin of the packet i.e. the Tracker. Have added a note at the top. I know it's a bit confusing so open to ideas!

[S57PNX]

@madpsy that's fine, but Tracker S59HIJ-7 is geographically not in Croatia, neither is S58W-13 as an igate.

I suspect there is a glitch in translating coordinates to countries.

[madpsy]

Ah ok - there's maybe a bordering issue with the naturalearthdata.com dataset or a bug in my implementation. It's really difficult to get right as I've learned whilst doing this.

This uses the same location service which I wrote using that data - it's a lot more complicated to do than I originally thought!

https://lora-aprs.live/plotter.html

[S57PNX]

Yeah, the plotter has the same offset. S59HIJ-7 is in reality around 4km away from the Croatian border, yet your service shows it in Croatia. I did some spot checks around the country and there seems to be a random 0-4km offset from the real border. The nearest city feature works reasonably OK, so I simply believe that the border data is bad.

[madpsy]

That should be a bit better now, though won't be perfect due to the resolution. Hopefully improved though.