jgaehring / farm_offline

A FarmOS module for offline capabilities.
0 stars 1 forks source link

Roadmap #2

Open mstenta opened 6 years ago

mstenta commented 6 years ago

This module came out of discussions with @jgaehring and myself in this issue: https://github.com/farmOS/farmOS/issues/37

The idea is to create a fully browser-based solution to the problem of offline access to the farmOS application, using Javascript and service workers. We decided to start this module as a place to experiment with those ideas.

If this approach makes sense, it may become the starting point for an offline-first UI in the farmOS 8.x-2.x branch (on Drupal 8).

Rough plan:

The fourth step ("Provide some basic features") is obviously the biggest piece, effort-wise, and should be broken up into separate tasks once we get the initial proof-of-concept working (steps 1-3).

jgaehring commented 6 years ago

For the last 2 days I've been familiarizing myself with the Drupal/farmOS environment, and re-familiarizing myself with the Service Worker spec.

First, the good news: service workers get along very well with the Drupal architecture! Just about as well as any SPA, to the extent that I think we no longer need to think of the server-sidedness of Drupal as any kind of drawback where SW's are concerned. With the bit of boilerplate @mstenta provided me, I was able to get a SW registered in pretty short order and cache a few static assets to the browser's storage.

It doesn't matter whether it's a SPA or not because of a neat design feature of the service worker which I had forgotten. As long as the service-worker.js file itself (the one that gets loaded into the browser) is placed at the website's root directory, it can control all other resources in that directory and all its subdirectories. The file that points to service-worker.js and tells the browser to register it can be at /farm/assets/plantings/farm_offline.js or anywhere and everywhere in the site, but they will all share a single service worker, and the service worker can cache all endpoints and all their assets, no matter what page initially told the browser to register it. So cool!

Where Drupal starts to complicate things is when it comes to telling the service worker what resources to cache, because Drupal generates loads of urls and they're all potentially dynamic.

But that brings me to the second piece of good news: the Progressive Web App module handles all this marvelously. The module provides all the necessary urls to the service-worker.js and then makes sure that script is registered at root. Not only that, but it targets almost all (and I'll get to that 'almost' part below) of the offline use cases we have, and without any extra bloat. Apparently this module has been proposed to be included in Drupal Core, and for good reason; the author, nod_, is the JavaScript maintainer for D7 and D8 and he's used some very good patterns in developing the SW. I added the module to farmOS and it ran like a charm, with a few hiccups (upon enabling it I got the errors mentioned here, but it still worked). Since I'm only testing this locally I had to use devtools to mock 'offline' use, and it definitely needs real-world testing, but right out of the box it's caching the entire site and most of its assets (images are not cached by default, but that can easily be reconfigured). Looking at it now, I'm not sure we need to relegate this functionality to a special div like we originally discussed. If it tests well, it may be easiest to just let it loose on the whole site, rather than create extra work to confine it.

So this guy's pretty much done all my work for me. Except for that 'almost' part I mentioned above. The one use case this library doesn't address is the most significant one, which is handling offline PUT requests and caching their payload until connectivity is restored. And he excludes that feature for good reason, as I've learned, because it relies on a part of the Service Worker API which is non-standard. It is supported by current versions of Chrome and Firefox, and Edge has it behind a flag, but until it's on a track to become a standard, it should be considered unstable.

Given that farmOS' users can be given the choice to opt in on experimental features, I'm still game to try to develop this and see where it goes. And this might be where the idea of an overlay div or something similar might come back into play. Another possibility might be to reroute those requests to something like an /outbox page where the user can view updates that are still in cache but haven't been pushed to the server yet (and this reroute is possible, in theory, because the service worker lives at root). And if we still want to pursue the SPA idea, this could be the seed for that. So in other words, nod_'s PWA module would handle the offline read requests, while the (experimental) farm_offline module would handle offline write requests.

At least, that's my perspective on it so far. Does that make any sense?

mstenta commented 6 years ago

This is SUPER exciting! I'm really looking forward to trying out the PWA module and seeing what it provides out of the box! And yea I agree with your assessment that the "storing submitted forms locally when offline" is the most complicated piece - but I think we can take small steps forward - start simple and go from there!

So maybe it makes sense to add a dependency on PWA to the farm_offline module? Want to take a look at it together sometime and assess where the edges are?

mstenta commented 6 years ago

Another possibility might be to reroute those requests to something like an /outbox page where the user can view updates that are still in cache but haven't been pushed to the server yet (and this reroute is possible, in theory, because the service worker lives at root). And if we still want to pursue the SPA idea, this could be the seed for that. So in other words, nod_'s PWA module would handle the offline read requests, while the (experimental) farm_offline module would handle offline write requests.

I love this idea. :+1:

jgaehring commented 6 years ago

Yes, let's go over this soon. Meanwhile I'm going to keep reading up and experimenting with my install. I'm not sure how closely integrated this needs to be with nod_'s module to work. Typically all service worker logic gets bundled into 1 sw.js file, which would mean our module would have to work pretty closely with his. However, the convention for background sync seems to be that you register a "one-off" service worker, which removes itself after the syncing has succeeded. In which case our module could function rather independently from his. Might also be worth considering writing the background sync module separate from whatever UI logic we write to control it (ie, the SPA or modal). But these are all architecture issues I will probably grasp better as a I learn more. Perhaps we can talk next week after I've done my homework.

alexadamsmith commented 6 years ago

Hi @jgaehring - I was reading through your discussion with @mstenta about creating an offline mode for farmOS site. The project looks really promising and interesting! it also parallels a project that I've been working on - creating a native front-end app for farmOS. The end goal is pretty much the same - allowing access to previously stored records and create new records while offline, then synching to the farmOS server when the user goes online.

My initial approach involved building a native iOS app in Swift which communicated with the Drupal 7 RESTful web services module. This was going pretty well - I'd established user authentication, could request records from farmOS, and could post new records. But after a recent conversation with @mstenta, we decided to change course. We'd like to try making a hybrid app that is written in HTML+JavaScript, and packaged to run as a native app using the Cordova framework. Cordova provides a basic web view that lets a JS app run in a native environment, and also gives the app access to device features such as native memory, the camera, and geolocation.

We decided this would be the best approach because

Since I'm not an experienced JavaScript developer, I have more learning to do before I can start building a hybrid front-end app for farmOS. And before I get started, I would greatly appreciate the chance to talk with you about any areas where our projects might overlap. I know very little about service workers, but I expect that we might encounter some common tasks and challenges.

From my perspective, it seems like a native app could serve as a great short-term offline solution for farmOS, with the long-term goal being a totally browser-based experience using service workers. As I work on the farmOS front end app, I would like to build it with an eye towards a fully browser-based future when the farm_offline module is able to become a standard feature!

Anyhow, I'd love to talk more and identify places where our projects might overlap!

jgaehring commented 6 years ago

Yes, I'd love to talk! Sounds like a great path. Mike and I talked a little about the current landscape of hybrid frameworks. I haven't used any of them before but I've been hearing a lot about them.

It would definitely be good to coordinate our approaches, so the offline experience is as consistent as possible between browser and native. I'd also love to pick your brain on the REST module, since I haven't tried it out yet. I've been playing around with a standalone demo to make sure I understand how the service worker should work before I try integrating it with Drupal. I'm just sending POST requests to a mock server for now but I'll need to know how to format that properly when it connects to Drupal. Perhaps you'd have some pointers for me.

Hit me up as gaehj457 at the ol google mail service and we can work out a time to chat.