Speed up time to first contentful paint

[laurenwalker]

MetacatUI currently has an embarrassingly high time to first contentful paint. Testing knb.ecoinformatics.org results in a whopping 3.6 seconds from initial page load to first contentful paint.

The symptoms

• Users are waiting several seconds for the content to show up on the page.
• The initial loading icon is shown for that 3.6 seconds, which is annoying and doesn't give feedback about what the page is doing
• As the view is rendered, there is a lot of layout shift

The causes

• JS is not minified or uglified (see #224)
• Modules (views and models) are too large with too many dependencies and the app dependency graph is too convoluted.
• Most importantly, the application is front-loaded. Too many unnecessary modules are fetched as soon as the app is initialized rather than only when they are needed.

My recommendations

• Manage dependencies via ES6 modules rather than RequireJS. RequireJS was useful at a time when browser's didn't have great dependency management, but with browsers supporting Javascript modules, we should take advantage of that rather than use an external library. This will remove several Require-related file dependencies. It also paves the way for...
• Bundle app assets into modules via a bundler tool like Webpack or Snowpack. This will also minify and uglify the JS.
• Minify and bundle the metacatui CSS
• Bundle the app config with the AppModel rather than fetching it on page load
• Refactor the dependency graph so that only what is needed is fetched at first and a lean skeleton view of the app is rendered on the page while other assets are retrieved.
• Move View templates to the View JS files rather than separate HTML files. Consider bundling view-specific CSS with views, too. This is a more React-like web component approach.

Performance breakdown with specific recommendations

I tracked the breakdown of network requests made by MetacatUI before contentful paint. Here is the timeline.

Initial app setup (240ms-800ms]

• In sequential order: config.js, loader.js, theme config (again?), require.js (takes 230ms), favicon, app.js
  • Recommendations:
    • Refactor this setup process so that these steps don't have to be taken in sequential order. Create a build process for MetacatUI that bundles the main config and theme config together. Bundle the loader.js and app.js together.
    • Remove the dependency on RequireJS in favor of ES6 modules (GH issue to follow)

App scaffolding [800ms-2.8sec]

• Then the AppView and AppModel are fetched and initialized from app.js. Bootstrap is also requirred at this point, which has jQuery as a dependency so jQuery is fetched.
  • jQuery takes 250ms
    • Recommendation:
      • Remove jQuery completely (big refactor) or upgrading to 3.6.0 shaves off 10KB
      • Question whether Bootstrap is needed at this point in the app - can this be defered?
• Dependencies of AppView and AppModel are retrieved:
  • Subviews:
    • Navbar, AltHeader,Footer, SignIn. - These take about 300ms total and take up 4/6 available requests to the host.
    • Recommendation:
      • Lazy load Footer after user scrolls down page.
      • Render a lean skeleton AppView that renders immediately at this point and then grabs the subviews later, so at least the user sees something on the page at this point.
  • JS libraries:
    • Require-text library is retrieved for RequireJS to require template HTML files - 350ms
      • Recommendation:
        • Remove RequireJS dependency in favor of ES6 modules.
        • Consider a web component approach to templating by bundling the template HTML with the view itself in the same file. Themes can override templates in the config.
    • UnderscoreJS and bootstrapJS (250ms for both, retrieved at same time)
      • Recommendation: Consider removing Underscore as a dependency as most of it's utility has been replaced by ES6 features
    • BackboneJS
    • These take 350ms total and take up 4/6 available requests to the host
  • HTML Templates:
    • alert, appHead,jsonld.txt, app,loading,altheader,defaultheader, navbar, footer, login, loginbuttons, loginoptions, logindefault
    • These take 250ms total and are 13 individual requests
    • Recommendation:
      • Consider a web component approach to templating by bundling the template HTML with the view itself in the same file. Themes can override templates in the config.
      • OR Lazy load templates that aren't used until after an event. (e.g. login only needs to render when user clicks login).
• All Views and Models that are stored globally are required at this point by App.js. This kicks off a huge process of loading a massive dependency graph of models and views, most of which are not immediately needed by the app.
  • Recommendation:
    • Don't create global instances of modules until they are needed. Consider moving these references to a centralized controller that gets/sets them.
    • Minimize top-level dependencies by lazy-loading modules and breaking down big Views into small components.
• Two fetches are made immediately from global models:
  • Dataone auth token
  • Dataone Node Info
    • Recommendation: Investigate why this is retrieved right away. See how it can be lazy loaded. Consider caching node info.

Header and Footer render [2.8sec-3.4sec]

• All global views and models are now fetched, so now app.js tells the AppView to render(). This should be happening way efore 2.8sec.
• CSS
  • 6 CSS files are loaded first (6 is max concurrent requests to the domain) and are blocking rendering of page:
    • THREE Fancybox CSS files are render blocking
    • metacatui-common is 215ms: Minify metacatui CSS files (metacatui-common is 191 kb)
    • FA is 89ms: Switch font-awesome to modules, where there's a CSS file per icon and overall CSS file is smaller. Could load FontAwesome later in the rendering flow since it's not criitical
    • Bootstrap is 215ms - possibly bundle with metacatui css?
  • Next CSS files: (~250 ms)
    • 2 theme css files
    • metacatui-common responsive css
    • bioportal css
  • Recommendation:
    • Minify and bundle all critical metacatui css into one file.
    • Separate View-specific CSS into their own files and require only at time of View render. (Use a React-like web component approach)
    • Remove fancybox CSS and use vanilla JS instead (I think this is used for the gmap modal?)
    • Defer FontAwesome until app is done rendering. Critical icons can be embedded as SVG
• Images
  • 4+ View images: theme nav bar/header images (KNB flower img, ADC mountain image), navbar theme logo, footer logos
  • 10 (yes, 10) favicon.ico images are retrieved next
    • Recommendation: Restructure appHead.html to only include the favicon needed for the current browser.
• Fonts
  • FontAwesome, Lato/Aller (KNB/ADC)
• At this point the header and footer is displayed on the page, including images and CSS styling

Content render [3.4sec-3.6sec]

• At this point the router figures out which view the user has navigated to and the AppView starts rendering that parent view (DataCatalog, Profile, etc)
• Time to first contentful paint: 3.6 seconds (!!)

Layout shift screenshots

This timeline causes the following layout shift. Here's the KNB theme as an example:

[robyngit]

Just to supplement this, here are results from a light house test for the main KNB landing page and the KNB search page. Lauren's recommendations will have the most impact, but there are some additional small, easy tasks that we can start with like preloading important images.