Look into ServiceWorker again for serious

[chrisblakley]

Look into the feasibility of implementing ServiceWorker into Nebula. Not 100% sure what I'd do with it, but it'd be the last step before every Nebula site could become a true progressive web app.

https://justmarkup.com/log/2016/01/add-service-worker-for-wordpress/

I can already foresee difficulty because of where the sw.js file needs to live outside of the theme directory...

[chrisblakley]

Could a Service Worker be used to help with this? https://github.com/chrisblakley/Nebula/issues/1342

Or maybe Fetch or the Cache API? I want to load the page, then get the following CSS files after the first meaningful paint, but still have them cached in the browser for future pageviews:

• Mmenu CSS
• jQuery UI CSS
• Font Awesome CSS

[chrisblakley]

This will also enhance the HTTP2 Server Push so that it can become cache aware.

[chrisblakley]

May need to provide a sw.js file in the /resources directory with instructions on how to implement. Then, maybe a Nebula Option when that has been done?

[chrisblakley]

Ok, I've got a service worker working on Nebula for testing. Seems like it could be really useful, but I have a few questions/concerns to figure out.

• The sw.js file must live in the root directory, so in order to do that we have 2 options (that I can think of):

  • Like I mentioned above, we could bundle it in the /resources directory and direct the developer to move it to root.
  • We could approach it like we do the manifest.json file and actually code the sw.js file in PHP and write to the root directory.

• I need to figure out how to tell the sw.js what files to cache.

  • It seems that the cache files array must be hard-coded into the sw.js file before the install event...
  • If we were to write the sw.js file via PHP, we could then tell it which files to cache (by looping through enqueued styles and scripts.
  • Perhaps the files to cache could be told to sw.js after the install event by passing it via JS? That may feel less hacky, and we could also pass it images too.

• To tell sw.js to "uncache" files when we update them, we must update the cache name... This could be a pain point when developing or making changes.

  • If we were to write the sw.js via PHP this would also be easier to maintain.

Seems like writing the sw.js file from PHP has a few benefits, but it just seems really hacky. I want to do more research before making this decision.

[chrisblakley]

Here's how far I got today:

main.js (inside window load):

console.log('window load 2');

//ServiceWorker
//Note: This can also be used to preload pages. Maybe when a user hovers over a link we can use that idle time to preload the next page?
if ( 'serviceWorker' in navigator ){
    console.log('serviceWorker supported');

    navigator.serviceWorker.register(nebula.site.directory.root + '/sw.js').then(function(registration){
        // Registration was successful
        console.log('ServiceWorker registration successful with scope: ', registration.scope);
    }, function(err) {
        // registration failed :(
        console.log('ServiceWorker registration failed: ', err);
    });

    //Cache API... Does this go here or in sw.js?
    if ( 'caches' in window ){
        //Some calls to the cache API should be added to the conditionalJSLoad() function so that when they are loaded that way, they can be cached!
        caches.open('nebula-test-cache-v0.0.3').then(function(cache){ //Since there can be multiple caches, the cache name must match what is in sw.js!
            //Add a single resource to the cache:
            console.log('adding docs CSS to cache');
            cache.add('https://gearside.com/nebula/wp-content/plugins/nebula-docs/nebula-docs.css');

            //Add a bunch of resources to the cache:
            //Note: if any resource cannot be reached, none will be added to the cache!
            urlsToPrefetch = [
                'https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/4.0.0-alpha.6/css/bootstrap.min.css',
                'https://cdnjs.cloudflare.com/ajax/libs/jQuery.mmenu/6.1.0/jquery.mmenu.all.css',
                'https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css'
            ];
            cache.addAll(urlsToPrefetch.map(function(url){
                return new Request(url);
            })).then(function(){
                console.log('Small test resources have been fetched and cached.');
            });

            //Testing adding all enqueued styles/scripts to the cache...
            //Note: If this is requesting all the files first, then doing this is counterproductive! It'd be best to add them to the cache API when they are called dynamically in Nebula's conditional JS/CSS function.
            //Perhaps during idle time we can prefetch the ones that haven't been used yet? I'm still trying to determine what best practices are for that stuff.
            /*
                cache.addAll(Object.values(nebula.site.resources.js).map(function(url){
                    return new Request(url);
                })).then(function(){
                    console.log('All enqueued JavaScript resources have been fetched and cached.');
                });

                cache.addAll(Object.values(nebula.site.resources.css).map(function(url){
                    return new Request(url);
                })).then(function(){
                    console.log('All enqueued stylesheets have been fetched and cached.');
                });
            */
        });
    }
}

sw.js:

var CACHE_NAME = 'nebula-test-cache-v0.0.3';
var CACHE_FILES = [
    '/',
    'https://gearside.com/nebula/wp-content/themes/Nebula-master/assets/img/phg/phg-nebula.jpg', //You can use absolute URLs or relative paths (to the registerred route)
    'wp-content/themes/Nebula-master/assets/img/phg/phg-nebula-mountain.jpg',
    'wp-content/themes/Nebula-master/style.css',
];

//Install
self.addEventListener('install', function(event){
    console.log('[ServiceWorker] Installing....');
    event.waitUntil(
        //open this cache from caches and it will return a Promise
        caches.open(CACHE_NAME).then(function(cache){ //catch that promise
            console.log('[ServiceWorker] Caching files');
            cache.addAll(CACHE_FILES); //add all required files to cache it also returns a Promise
        })
    );
});

//Activate
self.addEventListener('activate', function(event){
    console.log('[ServiceWorker] Activate');
        event.waitUntil(
            //it will return all the keys in the cache as an array
            caches.keys().then(function(keyList){
                //run everything in parallel using Promise.all()
                Promise.all(keyList.map(function(key){
                    //if key doesn`t matches with present key
                    if ( key !== CACHE_NAME ){
                        console.log('[ServiceWorker] Removing old cache ', key);
                        return caches.delete(key);
                    }
                })
            );
        })
    );
});

//Fetch
//Note: What is it about only being able to fetch once? I know we can clone stuff, but why would we need to (what benefit is there to need to call a file more than once)?
self.addEventListener('fetch', function(event){
    //note that event.request.url gives URL of the request so you could also intercept the request and send a response based on your URL (e.g. you make want to send gif if anything in jpeg form is requested)
    console.log('[ServiceWorker] Fetching...');

    //it either takes a Response object as a parameter or a promise that resolves to a Response object
    event.respondWith(
        //If there is a match in the cache of this request object
        caches.match(event.request).then(function(response){
            if ( response ){
                console.log('[ServiceWorker] ******* Fulfilling " + event.request.url + " from cache.');
                return response; //returning response object
            } else {
                console.log('[ServiceWorker] ' + event.request.url + ' not found in cache fetching from network.');
                return fetch(event.request); //return promise that resolves to Response object
            }
        })
    );
});

Fighting cached files can be tough too especially when main.js gets cached. Need to find a way to combat that (maybe with ?debug).

Check your Application tab under Service Workers in Chrome dev tools to be able to modify what is active.

[chrisblakley]

Posted some questions here: https://www.reddit.com/r/learnjavascript/comments/6jfb94/some_questions_about_serviceworker_and_caching

To recap them here:

1.) Why am I declaring the files to cache inside the sw.js file? It seems to me like it's the utility and should be told what files to cache from the main.js file that is registering it. Which leads me to my next question...

2.) I found that I can use cache.add() and cache.addAll() to make my sw.js file load resources from the cache that were not declared within it. Seems great, but one note I saw in a tutorial said that this requests the resource first and then caches it... So I'm guessing I should not just loop through every resource my site could need and add them all to the cache.

3.) If I do load a resource (through a tag or something) and then use cache.add() for it later, does it load the resource again at that time, or use the one that is already loaded? What about if I load a JS file over AJAX (with caching enabled)?

4.) Is there any easier way to clear the cache than opening the sw.js file and changing the cache name? Is that what other developers/content managers are doing is constantly editing the sw.js file? What's tough is that if I do go the route I mentioned in question 2, then my cache name has to perfectly match between my main.js and sw.js files which seems risky to me that it may not always happen.

5.) I've see recommendations of not storing too many files in the cache... Is anything I've mentioned in these previous questions indicative of putting too much in the cache? How do other websites work with offline mode if they aren't putting a ton of resources in the cache?

6.) I've also been able to get HTTP2 Server Push working on my server which is really cool and I'm told ServiceWorker allows for huge speed improvements when combined with this method. Will these two things just go together naturally when I figure out the above caching questions, or is there additional implementation required?

7.) The Google tutorial was saying something about only being able to request a resource from sw.js once unless you clone it. None of the other tutorials mentioned this, but Google was sort of harping on it. What is a scenario where I would need to use a cached file from sw.js more than once? Seems like it's a 1:1 to me (at least at this moment of simply using it to speed up load times.

8.) I got basic offline working, and I'm serving a /offline/ page. However, when the network is available again, I'm still getting the /offline/ page because it became cached for whatever page was requested. My question: is cache.add(window.location.href); a bad idea in my main.js file? I'm guessing that's what is causing my problem. How do other sites cache the current page?

9.) When caching pages themselves, how do others deal with trailing slashes? I've set up htaccess to redirect all pages to have a trailing slash. However, when offline, some links do not have the trailing slash which means when it is clicked the user gets a network error page. Is it bad practice to add a trailing slash to all navigate request modes? Ideally there would be a more flexible solution in case the htaccess 301 gets changed.

[chrisblakley]

I've got Nebula to a point where it works offline now.

https://gearside.com/nebula/

Open dev tools, go to the Network tab, check "Offline" and refresh (not a hard refresh). You should get the same page without a network offline error!

Some things I'd like to learn how to do when offline:

• Capture data in Google Analytics, and then send it all when the network becomes available again.

I recommend that we make an /offline page in WordPress (maybe automate this when the future Nebula Option "ServiceWorker" is added) and add it to the install cache within sw.js.

Also this is really helpful: https://jakearchibald.com/2014/offline-cookbook/

[chrisblakley]

One tutorial recommended writing part of the sw.js file via PHP, so I know at least some other websites do that!

add_action('save_post', 'update_sw_version');
function update_sw_version(){
    //writes sw.js file with a new value for "version" variable
}

I also like the idea of hooking into the save_posts action to update the service worker file.

[chrisblakley]

As much as I hate the UX of this site, it's also an excellent resource: https://serviceworke.rs/

Find the "recipe" on the left column, then look in the far upper right corner for the different file snippets. If you want to copy/paste anything be careful because you'll select the explanation text on the left column for some reason.

[chrisblakley]

I got my Service Worker to tell all clients when it is online or offline. Here's what I'm doing:

//Check if we are offline
function offlineCheck(event, know){
    if ( know ){
        console.log('[SW] No need to check, we know we are offline.');
        networkPostMessage(event, 'offline');
    } else {
        console.log('[SW] Going to check if we are offline...');

        fetch('').then(function(){ //This actually works detecting online vs offline with an empty string in the fetch... Might actually leave it like that
            networkPostMessage(event, 'online');
        }).catch(function(){
            networkPostMessage(event, 'offline');
        });
    }
}

//Tell all clients we are offline
function networkPostMessage(event, availability){
    console.log('[SW] We are ' + availability + '. Sending message to client...');

    self.clients.matchAll().then(function(clientList){
        clientList.forEach(function(client){
            client.postMessage({
                client: event.source.id, //@todo: source is null for some events. Do I need individual IDs? Can't I just message all clients at once?
                message: '[SW] You are currently ' + availability + '!'
            });
        });
    });
}

No idea if any of that is stupid, but progress is being made. In main.js I have it adding a dashed border and changing the hero text when it detects offline. Eventually this will be controlled by the body class that it adds.

navigator.serviceWorker.addEventListener('message', function(event){
    console.log("Received a message from sw.js: ");
    console.debug(event.data);

    //Network Availability messages
    if ( event.data.message === '[SW] You are currently offline!' ){
        jQuery('body').addClass('offline').css('border', '5px dashed red'); //Border for reference
        jQuery('#hero-section h1').text('Offline Nebula');
        jQuery('#nebula-hero-formcon').addClass('hidden');
    } else if ( event.data.message === '[SW] You are currently online!' ){
        jQuery('body').removeClass('offline').css('border', 'none'); //Border for reference
        jQuery('#hero-section h1').text('Nebula');
        jQuery('#nebula-hero-formcon').removeClass('hidden');
    }
});

[chrisblakley]

This was all working great last night when I went to bed, and now on a different computer this morning I'm getting errors:

Uncaught (in promise) DOMException: Failed to execute 'waitUntil' on 'ExtendableEvent': The event handler is already finished.

This happens in at least 2 locations:

• On install at: event.waitUntil(self.skipWaiting());
• On activate at: event.waitUntil(self.clients.claim());

Edit: I was using Chrome Canary (v60) to test last night (where it was working). These errors are happening in regular Chrome (v59). While that's helpful, I still need to figure out how to fix this issue...

Ok, I think I didn't need the waitUntil() inside of the .then() since that was already inside a waitUntil(). I removed them and it seems to be working fine now.

[chrisblakley]

Another cool feature I was thinking about is that we could use this to fetch stuff from the server (like JSON) and push updates back to the client without needing refreshes and without bogging down the client with tons of AJAX.

There would have to be some kind of listener in the sw.js file (since it isn't server-side JavaScript), but we could tie in desktop notifications for updated data. The first thing that comes to mind is our intranet, but event-based websites could use this too.

Edit: a setInterval() inside of the service worker might not be reliable, but we could look into using a periodic sync here: https://github.com/WICG/BackgroundSync/blob/master/explainer.md

[chrisblakley]

When offline, look into storing things like Google Analytics gifs in an outbox and then background sync when connection becomes available again: https://github.com/WICG/BackgroundSync/blob/master/explainer.md

[chrisblakley]

Closing this since the initial push has been made. Any individual improvements will use their own issues.