[Website] 100% reliable deployments

[adamziel]

Playground deployments require clearing the cache and the service worker way too often. Let's ensure a high standard of stability. All deployments should always work on all browsers without having to clear the cache.

• https://github.com/WordPress/wordpress-playground/issues/1774
• https://github.com/WordPress/wordpress-playground/pull/1822
• https://github.com/WordPress/wordpress-playground/pull/1849

Done is

We have an E2E suite that tests a Playground website deployment from a very old version to a new version, and ensures the following things work:

• New webapp runs on the first page visit.
• Offline mode.
• Direct visits and iframe embeds.
• Neither cache layer should yield stale data (HTTP, service worker, OfflineCache, other Cache instances, etc.)
• The open tabs that can be reloaded without a data loss are reloaded. Other open tabs are left alone
• Safari, Chrome, Firefox, mobile browsers.

There should be no intermittent failures, stale fetch() responses, or problems with stale service workers.

Root cause of the problem

Two reasons are at play:

• Dependency graphs
• Caching

Dependency graphs

Deploying a new Playground version does two things:

• Publishes new assets on playground.wordpress.net
• Deletes old assets

If the previous version of Playground is still running, it will attempt to fetch the old assets – and fail:

This wasn't a big deal a few months ago, since a page reload would solve this, but then we've introduced the offline support in #1483 .

Caching

The offline support keeps a copy of all the accessed old assets until the new service worker is installed. This might take 24 hours or sometimes longer! During that time, visiting playground.wordpress.net would load the cached index.html file and the rest of the stale dependency graph from the previous Playground release. Since some files are only loaded on demand, we'd get a mixture of cached assets and network errors – effectively putting the app in an undefined state.

The solution

I’m 95% convinced we must always force Playground to switch to the new service worker.

However, this would require refreshing all the open tabs and would trash any temporary Playgrounds.

Therefore, we might have to store all Playgrounds in OPFS, even the temporary ones. To maintain good UX, we'd add a cleanup mechanism to hide the "stored temporary" Playgrounds after a regular page refresh, and we'd keep them visible after a page refresh triggered by a new Playground release. We could also add a “Recently archived” button to recover anything archived during the last 24 hours – cc @jarekmorawski for thoughts.

Solutions without a forced page refresh

I couldn't find any solution that would keep the Playground site working without a forced page refresh:

cc @brandonpayton @bgrgicak

[adamziel]

With #1822 and #1849 merged, this seems to be done. Let's reopen if we notice any other problems with website deployments.