Slow loading

[kgoba]

Due to a large number of world-wide tracked sondes, sometimes the app is very irresponsive since it seems to load and track the data from all sondes in flight.

It would be nice to have an option to limit the track list to a certain radius. The API already supports this via the "/sondes" request and distance parameter, however, tracker.js does not seem to use this feature, instead loading the list from "/sondes/telemetry" which does not have a distance filter possible.

Please consider limiting the unnecessary data traffic and CPU time to process sondes from the other side of the globe.

[darksidelemm]

This comes down to a cost tradeoff on the part of how we host SondeHub.

More complex queries (such as those using distance based limiting) mean more CPU time in the database queries, which costs more. It does mean less data transfer, but I'm not sure that compares to the CPU costs.

Right now, on page load the tracker downloads X hours of history (default 3 hours), then switches to a websockets-based connection for further updates. By supplying everything, that initial query can be cached easily, which lowers the query costs.

Changing this approach requires some pretty careful thought, and could result in other issues. What happens when the user pans the map? How do they 'fill in' the areas they are missing? Does this mean more radius-limit queries (expensive), or do they just start seeing new packets arriving via websockets?

I agree that the map can get laggy, especially if left open for a long time, but it definitely needs a lot of thought put into how to handle this properly.

[kgoba]

What about client-side filtering? The initial request would be unchanged, the servers would happily supply the whole list of recently active sondes, but the client then decides which of them to display and establish the live websocket connections.

I understand the UX issues popping up. However, this could be solved at least partially by graying/circling the inactive map area.

Depends who the user is, I guess. I mostly use this to actively track sondes in the corner of the world where I live, and even recover them. It really hurts when you're in the middle of the forest with a skimpy internet connection and freezing fingers, waiting for the page to finally load, so I can't be fully impersonal here.

[darksidelemm]

In that last situation, if you know the serial number you can go to sondehub.org/ which will filter to just one particular serial number.

[kgoba]

Anyway, it's not a pressing issue, just an inconvenience. Definitely thanks for all the great work put into this already!

[ZigiWalter]

And if client-side filtering is ruled out, maybe instead of filtering by the distance from a station, is it more efficient to do course-grain filtering, but one that is common to many users - such that caching is still possible?

For example, maybe filtering by (part of a) continent? To accommodate users that are located "in-between areas", the GUI can allow to select more than a single area-of-intereset. Also, the areas themselves can be partially overlapping. Is that any better than filter-by-distance?

[TheSkorm]

There's certainly some improvements that we can make - most of which are going to be driven by the frontend and require polling the /sondes endpoint to get the list of current sondes, subscribing to near by sondes via websockets and pulling down history data for sondes that the user pans to.

If we built the tracker from scratch we'd likely have done something like this before however when we moved from habhub we basically made endpoints that emulated habhub rather than rewriting the frontend.

Not sure when these changes will happy as it's a lot of work but it'll likely be needed soon if we continue our current growth :)

[LukePrior]

@kgoba have you found the experience any better when using the tracker with the new "Only show visible on sidebar" option?

[kgoba]

Thanks, looks like a promising update. I will give it a go on the field, that's what counts at the end. I hope there are no unforeseen gotchas like clicking on a sonde, which will cause the map to zoom in on it, which will cause it to be the only one in the list, etc. But perhaps it's workable. Also depends how much is happening under the hood - are the hidden objects still allocated processing time/memory in the browser?