Allow to set scenario option defaults globally

JPustkuchen commented 1 year ago

Context: Quite often some of the advanced scenario settings apply to every scenario and are "global". For example the hideSelectors are the same for each scenario, e.g. for Cookie Banners, Sliders, ... or you need a delay on each scenario, ...

Problem BackstopJS doesn't yet allow to set defaults for these scenario settings at the global level.

There are several issues already, asking for single scenario settings to be set globally, like

Expected behavior would be that a scenario uses a) The BackstopJS predefined defaults, if no global default is set b) The global default if it is set and not overridden in the current scenario c) The current scenario setting, if it is set (and thereby overriding all global settings)

Proposed solution: Allow to define scenario setting defaults at the global level. I think it would be totally fine, if the global level just uses the same keys. For example set delay just like debug. If we fear clashing variables, these could be put below a parent setting like scenarioDefaults: {} but I don't think that's needed.

There is already one scenario setting which allows this: viewports // An array of screen size objects your DOM will be tested against. This configuration will override the viewports property assigned at the config root.

It would be great, if that could be the same for all scenario settings!

This feature would really help a lot to safe duplicate code, a lot of time and potential copy / paste errors. It would make the configuration json a lot smarter.

Thank you very much for this great great project!

rynodivino commented 1 year ago

Not too familiar with Backstop so excuse any ignorance. Seems like maybe something simple could be done here: https://github.com/garris/BackstopJS/blob/master/core/util/createBitmaps.js#L37

Maybe something like this:

  configJSON.scenarioDefaults = configJSON.scenarioDefaults || {};
  configJSON.scenarios = configJSON.scenarios || [];
  configJSON.scenarios = configJSON.scenarios.map(scenario => ({
    ...configJSON.scenarioDefaults,
    ...scenario
  }))

I'm sure there are more efficient ways to do it - just trying to give a rough idea

fastfedora commented 1 year ago

A workaround until this is supported natively is to use a JavaScript configuration file and build the scenarios dynamically.

For example, create a backstop.config.js file similar to:

const defaultScenario = {
  "cookiePath": "backstop_data/engine_scripts/cookies.json",
  "delay": 50,
};

const scenarios = [{
  "label": "Home Page",
  "url": "https://dayoptimizer.com/",
}, {
  "label": "Tutorials",
  "url": "https://dayoptimizer.com/learn/",
}];

module.exports = {
  "scenarios": scenarios.map(scenario => ({
    ...defaultScenario,
    ...scenario,
  })),
  // ...rest of configuration...
}

Then run backstop test --config="backstop.config.js".

janwagner commented 1 year ago

A workaround until this is supported natively is to use a JavaScript configuration file and build the scenarios dynamically.

For example, create a backstop.config.js file similar to:
const defaultScenario = {
  "cookiePath": "backstop_data/engine_scripts/cookies.json",
  "delay": 50,
};

const scenarios = [{
  "label": "Home Page",
  "url": "https://dayoptimizer.com/",
}, {
  "label": "Tutorials",
  "url": "https://dayoptimizer.com/learn/",
}];

module.exports = {
  "scenarios": scenarios.map(scenario => ({
    ...defaultScenario,
    ...scenario,
  })),
  // ...rest of configuration...
}
Then run backstop test --config="backstop.config.js".

Aweseome!

AloisSeckar commented 1 year ago

Thank you for pointing me to the right direction, @fastfedora :+1:

JPustkuchen commented 6 months ago

I think this would still be a very helpful feature out of the box. Good to have a workaround, but it stays a workaround and is bad UX / DX. :(

dgrebb commented 6 months ago

@JPustkuchen thanks for putting together such a fantastic collection of details to start :)

@rynodivino, @fastfedora yay for healthy discussion in a good direction!

@janwagner, @AloisSeckar, @JPustkuchen, @garris Request For Comments here: #1539.

Mentioning #1309, #1090, and #715 for anyone tracking those issues.

Try it out:

npm install dgrebb/BackstopJS#feat/1432-global-scenario-settings

garris / BackstopJS

Allow to set scenario option defaults globally #1432