Incremental fetch: Persistent tracks with low overhead

tuffnerdstuff commented 3 years ago

I so far love the no-frills way of sharing my location with Hauk! When I shared my first bike track with my family, the only complaint I got was that "the tracks disappear after reloading". So I dug into the configuration and found out that you can adjust two properties to get longer persistent tracks:

max_cached_pts: maximum number of points stored in the backend for every user
max_shown_pts: maximum number of points drawn for every user in the frontend

Let's say you want tracks of max 1000 points displayed to the user, then you could set max_shown_pts to 1000 and max_cached_pts to 5 and you would get what you want as long as you do not refresh your browser. If you do so, then you will be presented with a track of only 5 points. That is because max_cached_pts also determines how many points (max) can be retrieved each time the frontend polls for new points via fetch.php. So the initial track can be at most max_cached_pts long, and subsequent points will be added until max_shown_pts is reached.

So far so good. Now I can simply set both max_cached_pts and max_shown_pts to 1000 and I will also retain my track (up to 1000 points) even after refreshing. The problem is that in the worst case (once the cache is filled) the frontend will retrieve 1000 points each update, which is set to each second per default. If you have lots of clients and/or lots of shares that might become a problem, but at best it is a bit wasteful.

Therefore I propose an incremental fetch:

fetch.php gets another optional attribute since, which limits the returned points to only those where point.time > since
if since is not provided, all points from the cache will be returned
The frontend can now use fetch.php without since for initialization ...
... and fetch.php with since derived from the oldest most recent point of all shares

That way you can have long, persistent tracks and minimal data overhead.

bilde2910 commented 3 years ago

That is a very good idea, and really makes a lot of sense - not sure why I thought it would be a good idea to send all the coordinates on every request, but your way certainly saves a lot of data by not having to transfer stuff that the client already knows about. Thank you a lot for your contribution!

tuffnerdstuff commented 3 years ago

That is a very good idea, and really makes a lot of sense - not sure why I thought it would be a good idea to send all the coordinates on every request, but your way certainly saves a lot of data by not having to transfer stuff that the client already knows about.

You can never anticipate everything up front. Given the default value of max_cached_points = 3 this would have never caused any trouble. Probably most folks don't have stalkers at home like I have and therefore are happy with short tracks :D

Thank you a lot for your contribution!

You're welcome!

bilde2910 / Hauk

Incremental fetch: Persistent tracks with low overhead #170