garris / BackstopJS

Catch CSS curve balls.
http://backstopjs.org
MIT License
6.66k stars 602 forks source link

ERR_ABORTED instability when running backstop test sequentially. #1506

Open galoliveira opened 8 months ago

galoliveira commented 8 months ago

Hi, I'm having trouble running backstop stably.

I currently have an app in Vue 2 and I created a script to authenticate onBefore and then take screenshots. I created a simple page with just the text ''hi'' to test, but I also used "complex" pages and both fail with the same error. The problem is that when running the tests sequentially (after passing, run again) the tests fail with the error below:

image

image

Note: in the error it searches for the name with two underscores (0__1), but the reference file that was approved has only one:

image

Any idea what it could be? Please help me, I've tried several alternatives and nothing works correctly. Thanks.

my json:

{
  "id": "backstop_default",
  "viewports": [
    {
      "label": "mobile",
      "width": 360,
      "height": 740
    },
    {
      "label": "web",
      "width": 1440,
      "height": 1024
    }
  ],
  "onBeforeScript": "puppet/onBefore.js",
  "onReadyScript": "puppet/onReady.js",
  "scenarios": [
    {
      "label": "Teste",
      "cookiePath": "backstop_data/engine_scripts/cookies.json",
      "url": "http://localhost:3000/teste",
      "referenceUrl": "",
      "readyEvent": "",
      "readySelector": "",
      "delay": 4000,
      "hideSelectors": [
        "#tex-toast"
      ],
      "removeSelectors": [],
      "hoverSelector": "",
      "clickSelector": "",
      "postInteractionWait": 0,
      "selectors": [],
      "selectorExpansion": true,
      "expect": 0,
      "misMatchThreshold": 0.3,
      "requireSameDimensions": true
    }
  ],
  "paths": {
    "bitmaps_reference": "backstop_data/bitmaps_reference",
    "bitmaps_test": "backstop_data/bitmaps_test",
    "engine_scripts": "backstop_data/engine_scripts",
    "html_report": "backstop_data/html_report",
    "ci_report": "backstop_data/ci_report"
  },
  "report": [
    "browser"
  ],
  "engine": "puppeteer",
  "engineOptions": {
    "args": [
      "--no-sandbox"
    ]
  },
  "asyncCaptureLimit": 4,
  "asyncCompareLimit": 20,
  "debug": false,
  "debugWindow": false
}