flood-io / element

💦Load test your app using real web browsers
https://element.flood.io
Apache License 2.0
354 stars 43 forks source link

Documentation on debugging wait timeouts #577

Open bessey opened 2 years ago

bessey commented 2 years ago

I am trialing Element for the first time today (2.X if it matters), and while in general it ROCKS there's one area I am struggling with: debugging a WIP test step. There's plenty of docs on the happy path by which I mean using the Browser / By / Until APIs, but what I have not found is what the heck you are supposed to do when e.g. an element is not found and the test times out, even though you believe it is on the page.

For example, given this test step

    step('Step 1: Navigate to Example', async browser => {
        await goTo(browser, '/example')
        // Prove it loaded
        const startLab = await browser.findElement(
            By.css('[data-qaid="button-start-lab"]', 'start lab button'),
        )
        await startLab.click()
        // Lab started
        await browser.wait(
            Until.elementIsVisible(By.css('[data-qaid="lab-quiz-form"]', 'lab quiz form')),
        )
        // Lab ready
        await browser.wait(
            Until.elementIsNotVisible(By.css('[data-qaid="lab-loading"]', 'lab loading bar')),
        )
        await browser.takeScreenshot()
    })

and this output

    ✖ Step 'Step 1: Navigate to Example' failed (32,772ms)
      frame.waitForFunction: Timeout 30000ms exceeded.
    at ElementVisibilityCondition.<anonymous> (/home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/page/Condition.js:111:29)
    at /home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/page/Condition.js:8:71
    at __awaiter (/home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/page/Condition.js:4:12)
    at ElementVisibilityCondition.waitFor (/home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/page/Condition.js:67:16)
    at Browser.<anonymous> (/home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/runtime/Browser.js:596:44)
    at /home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/runtime/Browser.js:17:71
    at __awaiter (/home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/runtime/Browser.js:13:12)
    at Browser.waitWithoutDecorator (/home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/runtime/Browser.js:577:16)
    at Browser.<anonymous> (/home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/runtime/Browser.js:138:24)
    at /home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/runtime/Browser.js:17:71
    at __awaiter (/home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/runtime/Browser.js:13:12)
    at Browser.wait (/home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/runtime/Browser.js:137:16)
    at Browser.<anonymous> (/home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/runtime/decorators/addCallbacks.js:31:44)
    at fulfilled (/home/bessey/.asdf/installs/nodejs/16.13.1/.npm/lib/node_modules/element-cli/node_modules/@flood/element-core/dist/src/runtime/decorators/addCallbacks.js:5:58) {
  name: 'TimeoutError'
}

How am I meant to deduce what selector is failing? I expected a stack trace to a line in the test file and perhaps even a function name, but I get no such thing. As far as I can tell there is no guidance on what to do in this situation in the docs.

Describe the solution you'd like

Better stack traces, but failing that, guidance in the docs on how to identify the failing line and steps you can take to debug.

Describe alternatives you've considered

I did try just scouring GH Issues for people who have had the same problem, but I had no luck.

bessey commented 2 years ago

I think in this case my issue is that await browser.wait(Until.elementIsNotVisible(...)) does not work how I expected it to. My hope was I could validate that an element that was in the DOM is no longer in the DOM, however I guess it's actually still supposed to find it, but for it to be hidden.

I am not sure there is a good pattern to validate an element you know was in the DOM earlier is not anymore.