TL;DR: This commit fixes the failure of the Newspeak service worker (sw.js) to update its caches. See description for (much more) details.

[mzimmerm]

• This commit fixes the failure of the Newspeak service worker (sw.js) to update its caches. This issue caused the Newspeak PWA client app on the user's device to never pick up server updates, notably the changed HopscotchWebIDE.vfuel.
• The changed files are sw.js, index.html, build.sh; there is a new script 'increase-pwa-version.sh' called from 'build.sh'.
• Apart from fixing the end-user experience, this change also touches Newspeak development (without any changes in the developer's workflow), as running '. ./build.sh' now changes the variable pwaVersion in 'sw.js'

• How to test this change:

  • Server / build - Get the latest version and make sure webIDE/index.html is copied to the webIDE directory where Newspeak is served locally for https://localhost:8080/webIDE, or for https://newspeaklanguage.org/webIDE. I am serving from the git version of webIDE, so I do not have to copy anything. - Run . ./build.sh as normally. - Make sure the updated webIDE/sw.js is copied into the webIDE directory where Newspeak is served. I am serving from the git version of webIDE, so I do not have to copy anything.

  • PWA Client / browser

    • You do NOT have to do anything, the versioned sw.js should now be installed, and use its versioned cache. But if you want to check: - In Browser, navigate to https://localhost:8080/webIDE, or https://newspeaklanguage.org/webIDE. - Open the PWA - In both browser and PWA, if you press F12 to debug (Chrome) and look at: Application -> Storage -> Cache Storage. you should see a versioned cache number, such as 'newspeak-ide-cache-version-13'.
    • Every time a new server version is build and the new build deployed, you should see a new cache version on the client PWA or browser, such as 'newspeak-ide-cache-version-14'.

Motivation for change:

• 'Before this change': After a new Newspeak version deployment on https://newspeaklanguage.org/webIDE, the PWA client app on the user's device did not update even after multiple launches, unless a manual cache clean was performed. I had the same experience (the PWA client not updating) with the webIDE server running on localhost, and the PWA client created from it. In fact, even the in-browser running webIDE did not update after server deployment.

• Having looked into how service workers function, it seems clear that the mechanism that causes the service worker update on the user's device (changing some bits) was never triggered. As a result, the service worker caches are never automatically updated, causing, among other things, the same (old) HopscotchWebIDE.vfuel to be used after server update.

Service worker lifecycle - summary

• From https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Tutorials/CycleTracker/Service_workers: - Once the PWA is installed on the user's machine, THE ONLY WAY to inform the browser that there are updated files to be retrieved is to change the service worker (this file, sw.js) on the server.

• Corollary: The only way for a PWA website to force an update of the service worker is to make a bit-change on the server's service worker (the sw.js).

  Such enforced update of the service worker, causes the browser to call a one-time 'install' and 'activate' on the service worker, which the service worker can use to delete old caches and cache the new assets.

Changes in more detail:

• index.html - moved caching of assets (cache.addAll) to the 'install' event in sw.js - added catching errors during serviceWorker.register
• sw.js
  • almost complete rewrite, although keeping the basics of using cache-first strategy (assuming used to lighten load on the server)
  • added a version number which should be changed on every deployment, to support the reinstall of the service worker on the client PWA browser. This number is increased automatically, during running each '. ./build.sh' by calling the added 'increase-pwa-version.sh'
    • 'const pwaVersion = 12;' - added a versioned cache name
    • 'const pwaCacheName = 'newspeak-ide-cache-version-' + pwaVersion;' - added a full list of assets - 'const pwaAppResources = [ .. see code .. ];' - added processing in the 'install' event: - all assets listed in 'pwaAppResources' are added to the NEW cache version 'pwaCacheName' - added processing in the 'activate' event: - OLD caches are deleted - modified processing in the 'fetch' event - In principle, this remains the cache-first fetching, but error checking was added etc - Assets not in 'pwaAppResources' would still cache here from network
• build.sh - added call to 'increase-pwa-version.sh' which increases version number in sw.js by 1, example - 'const pwaVersion = 12;' => 'const pwaVersion = 13;'
• increase-pwa-version.sh
  • a simple sed script that edits sw.js and increases version number by 1.

[mzimmerm]

Gilad, one comment inline

On Sat, Dec 23, 2023 at 6:11 PM Gilad Bracha @.***> wrote:

@.**** commented on this pull request.

In build.sh https://github.com/newspeaklanguage/newspeak/pull/124#discussion_r1435733318 :

@@ -11,6 +11,20 @@ pushd ../primordialsoup ./build os=emscripten arch=wasm popd

+# Back in 'newspeak'. +# Increase the pwaVersion in the PWA service worker, the newspeak/platforms/webIDE/sw.js +# This is required on every Newspeak build (strictly speaking, +# only on every Newspeak deployment to HTTP server), for the PWA clients +# to refresh all caches. +# Without the pwaVersion increase, PWA clients would not replace +# the cached HopscotchWebIDE.vfuel with the new version, +# unless forced by manual cache clean or similar + +. ./increase-pwa-version.sh

why is this ../increase-pwa-version.sh and not ./increase-pwa-version.sh?

I thought for sure the README.md recommended running the build.sh sourced as '. ./build.sh' .. although I am not able to find that now.

So I just followed the assumption and sourced the script as well: '. ./increase-pwa-version.sh'. I can make the script executable and execute it instead of source-ing it, although I would have to make a minor change: I took the opportunity it is sourced and slipped a return there in one branch, which would only work as sourced (although that branch is unlikely to run anyway, it is a ).

Let me know if you prefer to change the source-ing to executing.

Milan

— Reply to this email directly, view it on GitHub https://github.com/newspeaklanguage/newspeak/pull/124#pullrequestreview-1795505862, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALGQ7GVUL4O6NZYSE7UBE3YK5QKBAVCNFSM6AAAAABBATWUF6VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTOOJVGUYDKOBWGI . You are receiving this because you authored the thread.Message ID: @.***>

[gbracha]

I think this is fine.