Support of Progressive Web Apps (PWA) / Offline Sync

[jwillmer]

Currently the only possibility to create a PWA with offline sync is using PouchDB together with CouchDB. If I like to integrate this into Starcounter it will be a mess. A better way would be if Palindrom supports PWA based on JSON patch.

[tomalec]

Thanks for reporting that need. We were already considering writing Palindrom PWAs. Theoretically, JSON Patch (Versioned Queue & OT) should play well with offline behavior. We just didn't' start it before as we were focused more on already expressed customer needs, rather on our ideas of improvement and potentially useful new features.

Now, having your voice here, we have a reason to start it sooner than later. We will get back to you once we will have something to share.

[mmnosek]

Just quick note - together with @alshakero we talked with @jwillmer outside of this thread. Next step is to prepare some PoC by @alshakero so we can move forward.

[alshakero]

In the meeting we agreed to the following:

1. It’s not necessary to provide full functionality of the app in “offline mode”. However, some features (e.g. time tracking, filling up checklist forms) should work smoothly. The goal is to create rather simple applications than super complex offline systems. Starcounter is expected to support this on a product level.

2. Natural alternative and the candidate is a tandem of PouchDB + CouchDB that works as a client-side DB that synchronises with a central DB, Starcounter in this case. It may be used also as an inspiration for Starcounter. A disadvantage of this approach is having a stack separated from Starcounter, so any connection/synchronization is huge additional work. Using Starcounter would then makes less sense.

Theoretically, we have two ways of dealing with this:

• PouchDB + Starcounter
• Starcounter + (PWA Friendly) Palindrom. This approach is preferred by us to keep the client reasonably thin and not abandon all cool Starcounter features.

We would like to provide some kind of hybrid - make client only a little bit thicker in places where offline functionality is necessary. This would keep development effort reasonably low, provide full compatibility with other Starcounter Apps and fulfil PWA requirements. There are two major challenges with this:

• Caching the actual database data on the client side
• Design proper application flow and logic that doesn’t rely directly on the server as a single source of truth. Think validation and so on.
• ServiceWorkers only work in HTTPS.
• WebSockets are not supported in ServiceWorkers.

How to start

We decided that @alshakero is going to create a PoC of Palindrom with an additional ServiceWorker layer intercepting HTTP calls when there is no connection. Then we can test it with something real. Demo is expected soon :-) This will give us a much better understanding of the problem and how to solve it.

Other improvements in dealing with lost connection messaging etc. are obvious, to be reviewed later. Roadmap and overall (doable) idea is expected to be presented in the first half of January 2018 to make decisions.

This text is mostly wrttien by @mmnosek

[warpech]

The goal for us is to make all apps PWA-enabled. Network independence is one of 10 aspects of PWAs, and probably the hardest to implement.

It is up to the developer to decide whether he/she wants to build a thin view or a fat view. It should be possible to blend thin and fat views. The difference between the thin and fat views:

• a thin view is a view that does not have application logic on the client side. It will be static and read-only when offline.
• a fat view is a view that has client-side logic. To make a fat view operate when offline, the app developer provides a Service Worker that handles the CRUD operations performed by client-side logic. This could be for example done using REST calls.

Steps needed to achieve that:

Stage 0. Make Level1 bootstrapping HTML (PartialToStandaloneHtmlProvider.cs) cacheable by removing the session URI from it. The bootstraping HTML must be the same for app requests, including PWA. (https://github.com/Starcounter/Home/issues/333)

Stage 1. Create a Service Worker for Palindrom that (https://github.com/Palindrom/Palindrom/issues/185):

• Creates snapshots of view-models based on URL,
• When offline, respond with a patch that morphs the view-model to the snapshot based on current URL, so it covers URL-based navigation, not granular data changes,
• Maybe make the view-model read-only when its served from cache,
• Indicate when the app is actually offline.

Stage 2. Preload view-model snapshots to the app, so it can render URLs that were not rendered previously.

Stage 3. Allow app developers to provide Service Worker logic that handles view-model changes while offline, ideally without sacrificing blending capabilities. This is the difficult stage and some sacrifices must be made. We will have more meetings to discuss this. (Let's continue the discussion about this stage in https://github.com/Starcounter/RebelsLounge/issues/229).

[mmnosek]

Idea: Currently, by default all applications use PartialToStandaloneHtmlProvider.cs. To achieve some of the PWA capabilities (like making it available on home screen), more specific html is needed (different for iOS and Android). Shouldn't it be a separate middleware?

[alshakero]

Shouldn't it be a separate middleware?

We're working on that https://github.com/Starcounter/Home/issues/334.

[warpech]

Idea: Currently, by default all applications use PartialToStandaloneHtmlProvider.cs. To achieve some of the PWA capabilities (like making it available on home screen), more specific html is needed (different for iOS and Android). Shouldn't it be a separate middleware?

We're working on that Starcounter/Home#334.

Not really, since Starcounter/Home#334 is about overwriting the bootstrapping HTML by a system app. It should not be advised to an app developer to overwrite this file.

See the following line in another issue, https://github.com/Starcounter/Home/issues/333:

Please note that to achieve blending compatibility, the bootstraping HTML must be the same for all apps, including mobile apps, PWAs, etc.

It is a very good observation, @mmnosek, that there are many needs for custom content in the <head>. But currently there is no other solution to it than to add it dynamically: https://github.com/Starcounter/Docs/issues/178

Please note that this question is out of scope for Palindrom. If needed, please open a dedicated issue in RebelsLounge.

[warpech]

According to our updated roadmap, we will the first half of March on improving Palindrom reconnection once and for all.

After that, we will attempt to do an MVP of PWA, which can be summarised as:

• support for maintaining manifest.json in BlendingEditor (https://github.com/Starcounter/Blending/issues/170)
• primitive aggressive caching so that, when you're offline, you can still access what you have already seen (https://github.com/Palindrom/Palindrom/issues/185)

There are no plans of making any kind of solution for client-side logic for Palindrom blendable web apps. If there is a need for that, your best bet is a fat client web app + REST.

[warpech]

Since the term "PWA" is quite ambiguous (like "Web 2.0"), this issue will be broken down into actual feature issues.