feat: Offline Shopping List

michael-genson commented 1 week ago

What type of PR is this?

(REQUIRED)

feature

What this PR does / why we need it:

(REQUIRED)

This PR lets the shopping list work while offline. It does this by queueing requests, mirroring them locally, then sending them to the server whenever we can. There are a few caveats:

The user needs to first open the shopping list while they have connection. We're not prefetching the list before the user navigates to it. Once they've successfully fetched the list, they can lose connection
The local changes are only sent to the server if the user connects to the server while the shopping list is open. At that point all changes are sent to the server. There is no background task that does this automatically (e.g. if the user closes the app, more on this below)
If a user makes changes offline, and someone else makes changes online, and the user fails to reach the server within 5 minutes, their changes are considered "expired" and will be ignored (more on this below)

A few changes were made to get this to work:

The polling timeout has been increased to 24 hours. We don't know how long the user is going to be offline, but 24 hours is more than reasonable. We already disable requests if the user is idle, so we're still safe from bombarding the server with bad requests
Previously, there was a backend-triggered "Item Deleted Successfully" message upon item deletion. Since deletions are queued and sent later, this message doesn't really make sense anymore, so I got rid of it
To track state/reactivity, new items can accept an id from the frontend (so now the frontend generates a UUID before the item is even created on the backend)

Since not everything works offline (adding/removing recipes, changing the owner, etc.) we disable some features when we detect a network error (which will happen any time an item is created/updated/deleted, or during the normal 5 second polling period where we fetch the new list). I also added a warning banner letting the user know they're offline:

Which issue(s) this PR fixes:

(REQUIRED)

Fixes https://github.com/mealie-recipes/mealie/issues/3718

Special notes for your reviewer:

(fill-in or delete this section)

One implementation detail I simplified was the queue timeout. I mentioned that if the offline queue is too far behind the server (i.e. someone made changes a while ago) we just dump the queue. Ideally, we compare the timestamp of each item on the server, and only dump items with conflicts (e.g. if the offline user edits an item that an online user edited, dump that, but if the offline user edited an item that the online user didn't edit, keep that). However, this would be a lot more complex to implement (and probably be rather slow with large requests), and would probably only affect a handful of users. If this feature is requested later down the line we can consider adding it, but for now I think the existing behavior is fine.

I mentioned that updates are only sent to the server if the user is viewing the shopping list while online, and there is no background processing. PWAs do support this kind of background task using service workers, including using our nuxtjs PWA, (which fires even if the user isn't using the app), but hell if I can figure out how to get that to work.

Testing

(fill-in or delete this section)

Constantly testing the frontend online and offline. The online experience is unaffected (even though it's very different under the hood), aside from being a little more reactive (which is a nice plus).

michael-genson commented 1 week ago

For reviewers: in case you don't know, the Chrome dev console lets your throttle the network (including turning it off). Dev Console -> Network Tab

I strongly recommend anyone reviewing this to play around with the shopping list both online and offline, to try to find bugs. I found a bunch during development (which have been fixed)

boc-the-git commented 1 day ago

The things noted under Special notes for your reviewer, I think those are all reasonable decisions given we are already talking about a < 5% scenario (assumption) of the shopping list being used whilst offline, and they become edge cases within that 5%.

boc-the-git commented 1 day ago

All the code LGTM. I just want to do a bit more playing around with it before approving.

bartkummel commented 3 hours ago

Awesome, I didn't expect this to be fixed this fast, when I wrote #3718. I understand very well that things like these are quite complex. Thank you so much @michael-genson !

One remark, I saw the statement about this being a "< 5% scenario". While that might be true, fixing this makes a HUGE difference for all users that experience this. In this case "< 5%" doesn't mean that every user will experience this in 5% of the cases, but more like 5% of the users experience this all the time. So, in my case for example, it makes the difference of whether or not I can switch to Mealie (from Paprika), because without offline usage of the shopping list, I simply can't use it. My supermarket is on the ground floor of a larger complex. There's only a little bit of mobile data reception close to the entrance. The shop does offer WiFi, but that only works less than 50% of the time and also doesn't cover the entire shop.

Well, that's a very long message to just say how grateful I am. 😉 I can't wait until the upcoming release!

boc-the-git commented 3 hours ago

Awesome, I didn't expect this to be fixed this fast, when I wrote https://github.com/mealie-recipes/mealie/issues/3718. I understand very well that things like these are quite complex.

Michael being Michael 🤷

If you want to jump over to nightly @bartkummel you can use it immediately and give us first hand, real world feedback!

bartkummel commented 3 hours ago

@boc-the-git I'll see if I can squeeze some time out of my busy calendar this weekend.

mealie-recipes / mealie