Collaborative Data Patterns Catalogue

Table of Contents

• Technologies / dependencies
  • Built with
  • Front end:
  • Linting
  • Deployment
• Prerequisites:
• Project setup
• Design Tokens
• Component-centred design and the Fractal living styleguide
• Search integration
• Commands
  • Run dev server
  • Building for production
  • Building the living styleguide to static HTML
  • Secondary tasks
  • Linting
  • Building styleguide as static HTML
  • Check HTML links (WIP)
• Hosting, release and deployment
  • Hosting
  • Creating a new release
  • Deployment
• Project structure
• Configuration
  • Jekyll
  • Vue CLI
• Hacks
• Other useful bits

Technologies / dependencies

Built with

• Jekyll - static site generator
• Vue CLI - webpack-based build process
• Fractal Living styleguide
• git flow as a branching methodology
• Theo for managing Design Tokens

Front end:

• Sass for stylesheets (compiled with node-sass)
• Babel for JS transpilation

Linting

• Prettier
• ESLint
• Stylelint

Linting is enforced on a pre-push hook via Husky. This can be disabled / modified via package.json

Deployment

• Deployed and hosted with Netlify

Prerequisites:

• NodeJS runtime - ~10.x.x LTS release:
• Yarn
• Ruby 2.3.0 or greater
• bundler
• git-flow extensions (only required for release)

Project setup

git clone git@github.com:theodi/collaborative-data-patterns-catalogue.git
cd collaborative-data-patterns-catalogue
cp .env.example .env

Now populate Algolia environment variables in .env see Deployment, then

yarn install
bundle install --path vendor/bundle
yarn serve

Design Tokens

This project uses Design Tokens to manage shared values such as colours, font families, breakpoints and other size values, so that they can be shared easily accross projects and formats ((S)css, JS) while maintaining a single source of truth.

That single source of truth is held in Design Token files (spec) in the design/ directory.

The project uses the YAML formatting option for its design tokens for both ease of use and consistency with Jekyll metadata

Component-centred design and the Fractal living styleguide

This project takes a component-based approach to its design and development. Components are located in the assets/styles directory and are documented using the Fractal living styleguide tool.

Components are implemented in Fractal in the first instance, using the styleguide as a "workbench" to develop and test components and content scenarios for them before they are integrated into the Jekyll site.

NB As a Liquid templating adaptor was not available for Fractal at the time of development, the Fractal templates are written using the very similar but not identical Twig templating language

Search integration

Search is provided via Algolia, using jekyll-algolia to index the site at build time in Netlify, and Algolia's vue-instantsearch library to build the reactive JS search UI. Because of the static nature of the site, there is no fallback search for users without JavaScript.

Commands

All commands are via node package scripts.

Run dev server

yarn serve

This command does several things concurrently:

• Starts jekyll server with JEKYLL_ENV=development bundle exec jekyll serve --config _config.yml,_config.dev.yml on localhost:4000
• Starts a vue-cli / webpack asset server on a port defined in package.json under buildConfig.ports.assets
• Starts the Fractal styleguide on a port defined in package.json under buildConfig.ports.assets

You can run/debug each of these tasks separately via:

yarn serve:jekyll
yarn serve:assets
yarn serve:fractal

(the top level yarn serve task runs each of these tasks concurrently with one another)

Building for production

yarn build

• Compiles the design tokens via Theo
• Builds the revved assets via yarn build:assets
• Builds the Jeykll site via yarn build:jekyll / JEKYLL_ENV='production' bundle exec jekyll build

The built site is output to dist/, which is used as the web root directory on deployment

You can run/debug each of these tasks separately via:

yarn build:design-tokens
yarn build:assets
yarn build:jekyll

Building the living styleguide to static HTML

You can compile the Fractal styleguide to static HTML via

yarn build:fractal

This will build the styleguide to a styleguide/ directory in the project root

Secondary tasks

Linting

Linting (via yarn lint) is enforced on a pre-push hook via Husky. This can be disabled / modified via the husky key in package.json.

JS files are also linted at Babel compile time via eslint-friendly-formatter.

The following tasks are available for your manual linting requirements:

yarn lint:styles
yarn lint:js
yarn lint:html
yarn lint # all the things

Building styleguide as static HTML

yarn build:fractal

This builds the Fractal styleguide to static HTML and outputs it to a styleguide/ directory (gitignored) in the project root.

Check HTML links (WIP)

yarn lint:html

This runs html-proofer with options configured via a rake task.

This task is currently designed to be run locally (i.e. not in CI)

Because we're currently deploying the site to a subdirectory, it also requires some temporary changes to _config.yml or it will throw a large number of false positives:

Required:

• comment out the url and baseurl keys. This is required so that Jekyll writes out links in a way that html-proofer can resolve. This can probably be worked around using html-proofer's url_swap feature, but we have not currently got that working correctly.

Optional:

• Change env to development (not required but will disable html minification making the build much faster)

Hosting, release and deployment

Hosting

The site is hosted on Netlify, and is configured to build automatically on push to the master branch of the GitHub repo, via the Netlify GH app.

Creating a new release

A release script is included for convenience. Use a semver compliant version.

yarn project:release

This will:

• Prompt for the release type
• Create a new release branch from develop
• Bump the version key in package.json and commit it
• Merge back into develop and master
• Tag the release
• Push the release and tag to origin/master

NB Release script requires git-flow cli to be installed locally.

Deployment

Netlify is configured to automatically build the site on push to the master branch of the GitHub repo, via the Netlify GH app.

An environment variable of JEKYLL_ENV=production is set via the Netlify web UI in order to trigger Jekyll to use the production configuration when building the static pages for the site.

Additionally the following environment variables are set to enable the Algolia integration:

ALGOLIA_API_KEY
ALGOLIA_APPLICATION_ID
ALGOLIA_INDEX_NAME
ALGOLIA_SEARCH_API_KEY

Values for all these can be found in the Algolia dashboard under API keys, with the exception of the Application ID, which is listed in the sidebar of the Algolia UI.

The command run by netlify is:

yarn build && yarn algolia:index

Which unpacks to:

yarn build:design-tokens # Compile the abstract design tokens to scss/json
yarn build:assets # Build the Image, CSS and JS assets for the site via webpack / Vue CLI
yarn build:jekyll # Build the site's HTML via Jekyll
yarn algolia:index # index the built site in Algolia

Project structure

├── _config.dev.yml # Jekyll dev config - loaded in addition to _config.yml
├── _config.yml # Main Jekyll config
├── _templates/ # Generator templates for front end components - ignore!
├── assets/ # Front end assets - consumed by build process
│   ├── styles/ # Components for Fractal styleguide
│   ├── images/ # images - filenames will be revved on build so use rev filter in liquid
│   ├── js/ # JavaScript goes here, use ES6 as transpiled with webpack
│   └── docs/ # Markdown formatted documentation for Fractal styleguide
│
├── dist/ # Built site goes here
│
├── jekyll/ # The Jekyll part of the project
│   ├── _data/ # Data files in yml, primarily used for nav generation
│   ├── _includes/
│   ├── _layouts/
│   ├── _plugins/ # Custom plugins - currently just rev_filter for asset manifest parsing
│   ├── _templates/
│   ├── about/ # site content
│   ├── assets/ # Output dest for blendid build - ignored in git
│   ├── community/ # site content
│   ├── creating-impact/ # site content
│   ├── creating-open-standards/ # site content
│   ├── find-existing-standards/ # site content
│   ├── guide/ # site content
│   ├── index.md # Homepage
│   └── introduction /# site content

(Some files / dirs removed above for clarity)

Configuration

Jekyll

See inline documentation in _config.yml / _config.dev.yml for details.

Additionally, We are using the following Jekyll third-party plugins, auto installed via the :jekyll_plugins group in Gemfile.

• jekyll-git_metadata - Git metadata for last edited dates on oages
• octopress-minify-html - Minifies HTML on build only
• jekyll-sitemap - XML sitemap generator
• jekyll-seo-tag - SEO metadata generation

We also have the following custom Jekyll plugins (in jekyll/_plugins)

• env_vars.rb - exposes environent variables (inc from .env) to Jekyll
• rev_filter.rb - supports revving asset file names via a manifest.json file output by the asset build process

Vue CLI

The asset build process is configured / customised via the vue.config.js in the project root.

Hacks

• The jekyll-git_metadata is monkey patched in src/_plugins/git_metadata.rb to handle the fact that we're running Jekyll in a subdirectory of the project

Other useful bits

• Regenerate the table of content for this README.md (or any other) by running yarn docs:toc and pasting in the output