Resize before screenshot causing flaky tests

[aw-davidson]

When I run the debug window, I've noticed that there is some kind of resize happening before screenshots are taken. This is okay for most tests, but I also have certain scenarios with components which animate when resize is triggered. For these specific scenarios the screenshots are inconsistent and therefore the tests are flaky.

Is there any way to stop the resize? I'm currently looking through the source code for backstop but I don't see a specific resize happening. I think the resize event should happen before everything everything else is run so that the page has time to settle.

I'm running this in docker. My config:

var config = {
  id: 'myId',
  viewports: [
    {
      label: 'desktop',
      width: env_config.viewport.width,
      height: env_config.viewport.height
    }
  ],
  onBeforeScript: 'puppet/onBefore.js',
  onReadyScript: 'puppet/onReady.js',
  scenarios: [...myScenarios],
  paths: {
    bitmaps_reference: 'src/bitmaps_reference',
    bitmaps_test: 'report/bitmaps_test',
    engine_scripts: 'src/engine_scripts',
    html_report: 'report/html_report',
    ci_report: 'report/ci_report'
  },
  report: ['browser', 'CI'],
  engine: 'puppeteer',
  engineOptions: {
    waitTimeout: 90000,
    gotoTimeout: 300000,
    headless: true,
    args: ['--no-check-certificate', '--disable-dev-shm-usage', '--no-sandbox']
  },
  resembleOutputOptions: {
    ignoreAntialiasing: true
  },
  asyncCaptureLimit: env_config.baseURL.includes('http:') ? 1 : 3,
  asyncCompareLimit: 50
};

[garris]

Hi. This is puppeteer behavior-- not BackstopJS behavior. This might not happen if you specify a selector that is in the viewport.

[aw-davidson]

@garris Thanks for the reply. I'll try that.

[aw-davidson]

I was still seeing this behavior if I specified selectors that were within the viewport. My solution was to increase the viewport size and select the document instead. This seems to have stabilized my tests.

[garris]

Ah great! Nice problem solving there 👍

[veltrup]

I think this puppeteer-issue is relevant here 7251.

In my case it helped to insert this in the onReady.js.

await page.setJavaScriptEnabled(false);