search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.62k stars 8.22k forks source link

Visual regression tool #7523

Closed cjcenizal closed 6 years ago

cjcenizal commented 8 years ago

Goals

Our visual regression tool should allow us to visually document the state of Kibana's UI. It will help us understand:

  1. The scope of the UI
  2. How code changes are surfaced in the UI

    Interface

We should be able to run a script from the command line (e.g. npm run visualRegression) that explores various views and states of Kibana's UI, taking screenshots at each step, and then generates a gallery that highlights visual differences from a set of baseline screenshots. These baseline screenshots document how Kibana is supposed to look.

Implementation

We need to build a series of "functional tests" that only do the app-exploration and screenshotting work. They won't have any assertions, and they'll be completely uncoupled from the current functional tests. They'll be organized in their own folder and run via their own grunt task. This will be good for maintainability, since we'll be able to make changes to each set of tests without worrying about how they affect the other set.

Constraints

These tests will need to:

  1. Ensure all assets are loaded before running: CSS, fonts, and images (where possible).
  2. Document the app at various responsive breakpoints.
  3. Document the app in various visual modes, e.g. dark mode.

    Related tasks

LeeDr commented 8 years ago

The main problem I see with separating the screenshots from the functional tests is that the screenshots need to get to all the places and create all the visualizations and dashboard, and the functional tests already do that. The functional dashboard tests do load a .kibana index with one of each of the visualizations, so that could be a starting point. But it still seems like there would be a lot of duplication between the functional tests and the automation to get all the screenshots.

cjcenizal commented 6 years ago

Closing this in favor of relying on Jest snapshots as a proxy for visual regressions (https://github.com/elastic/kibana/issues/15282).