✅ Adds Percy.io Visual Testing to CI

bdolly commented 5 years ago

Runs visual regression testing on all builds and PRs

Motivation

We want to make sure that code changes made to isolated components don't have adverse effects on the whole system. Visual testing complements our existing unit and integration tests to verify that our entire UI–including components–aren’t just functioning correctly, but are also free of visual bugs or regressions.

Use Cases

On each commit we push to a PR we want to kickoff CI and compare a visual diff of our new stories to the base branches stories to ensure we changed all we intended and nothing that we didn't. This wil also help to scale the design system to a federated model so developers and designers alike can contribute with the security of knowing their contributions are tested and follow standards.

API changes

add npm script "snapshot": "yarn storybook:build && percy-storybook --widths=320,1280 --minimum_height=2000"

Implementation Notes

This adds several percy specific classes to hide non-essentail elements from rendering and being tested in percy snapshots.

@media only percy {
  /* set min-width for our stories so that components that are fill their container don't collapse */
  .percy-min-width{
    min-width: 960px;
  }

  /* hide addon-info header, props table, and description from Percy test */
  #root > div > div > div > div:first-child,
  #root > div > div > div > div:last-child,
  .hide-in-tests{
    display: none !important;
  }

  .show-in-tests{display: inline-block !important;}
  /* show our color swatches a11y table to check for changes */
  .ColorSwatch--a11y.hidden{display: block !important;}
}

Rendering and Storybook location

Percy picks up and renders every story in Storybook by default. 🚧 🚧 🚧 I have attempted to hide some stories by using the percy story parameter { percy: {skip: true} } on individual stories. However, this has not worked thus far and will be addressed with an upgrade to storybook v5 and percy v3 🚧 🚧 🚧

Functional Tests

Special test cases for percy only:

render all button props combinations
render color swatches a11y tables open to check for changes
set header and card stories to a min-width of 960px in percy tests so they don't collapse

dankolbman commented 5 years ago

UIKit Storybook for Review

Built with commit 7e4e20ca37c77bb1bd9e2fd6a2861e56e7394f0f

https://deploy-preview-98--kf-uikit.netlify.com

bdolly commented 5 years ago

Looks neat. I have a feeling we won't have a problem hitting the max snapshots per-month if we are actively developing, though.

paying for it would allow us to do mobile and IE regression testing though and save alot of worry on that front

kids-first / kf-uikit