reallymello
commented
1 year ago I forgot to mention I did try adding await in front of browser.waitUntil and/or the use of the command in the test and it does not help.
Open reallymello opened 1 year ago
reallymello
commented
1 year ago I forgot to mention I did try adding await in front of browser.waitUntil and/or the use of the command in the test and it does not help.
beatfactor
commented
1 year ago Ok, but if the command is async then await should be used.
reallymello
commented
1 year ago Agreed, but after going through all sorts of different iterations and not having expected behavior I started second-guessing how this command maybe was supposed to work with async await, but I guess it turned out to be this bug I'm reporting.
beatfactor
commented
1 year ago This is also related. @garg3133 has some more context.
garg3133
commented
1 year ago @gravityvi This does not seem to me like a selenium issue, but an issue with Nightwatch itself. I'm saying this because I did some fresh investigations and it seems that the working of waitUntil depends on whether the command written after waitUntil is chained together with waitUntil or written separately using await.
The following code works fine (the waitUntil condition function keeps on running again and again until the function resolves to true or waitUntil times out):
it('demo waitUntil', async function () {
browser
.navigateTo('https://ecosia.org')
.waitUntil(async () => {
const title = await browser.execute(function () {
return document.title;
});
console.log(title);
return title === 'Not the title I was expecting';
})
.findElement('input[type=search]');
});But in the following code, the waitUntil condition function runs only once and then the commands after waitUntil starts to execute, while the waitUntil condition function keeps on running in the background until true is returned or waitUntil is timed out.
it('demo waitUntil', async function () {
browser
.navigateTo('https://ecosia.org')
.waitUntil(async () => {
const title = await browser.execute(function () {
return document.title;
});
console.log(title);
return title === 'Not the title I was expecting';
})
await browser.findElement('input[type=search]');
});I've also updated #3511 with the new findings (added a new sample code in the issue description which works fine with async/await).
garg3133
commented
1 year ago And a workaround for @reallymello, if you chain the waitUntil in your custom-command with any random command from Nightwatch API, your test should work fine.
Below, I've chained waitUntil() with getTitle() command and it works as expected:
// nightwatch/commands/problemCommand.ts
export default class WaitForLoadScreen {
async command(this: NightwatchClient, expectedTitle: string) {
browser.waitUntil(async () => {
const title = await browser.execute(function () {
return document.title;
});
console.log(title);
return title === expectedTitle;
}).getTitle();
}
}@garg3133 thanks I will try the workaround. Can you also clarify since the custom command is async if the test using it needs to call it with await or if Nightwatch handles that for me?
For example, which is correct, 1 or 2?
it('demo waitUntil', async function () {
browser.navigateTo('https://ecosia.org');
browser.waitForLoadScreen(); // the async command inclusive of the workaround/fix
browser.click('#something`);
}it('demo waitUntil', async function () {
browser.navigateTo('https://ecosia.org');
await browser.waitForLoadScreen(); // the async command inclusive of the workaround/fix
browser.click('#something`);
}@garg3133 Yes, I observed the same and I'm using the same workaround in the Vite plugin. It is definitely an issue with Nightwatch, but looking at the waitUntil implementation everything looks as it should be, so the only thing that must be causing this is the queuing system.
garg3133
commented
1 year ago @reallymello No, you don't necessarily need to use await if a custom-command created by you is async, Nightwatch handles all that internally. So, both the examples provided by you are correct.
garg3133
commented
1 year ago @beatfactor Yes, it definitely seems to be an issue with the queuing system. What happens here is that when an async callback is passed to waitUntil command, as soon as the first execution of the callback resolves, waitUntil command resolves as well with some result and the commands following the waitUntil starts to execute, while the callback to waitUntil command keeps on executing in the background until it resolves to true. But this does not happen if the next command is chained to waitUntil.
beatfactor
commented
1 year ago I wonder if we could fix this for now by wrapping the entire waitUntil command implementation in a setTimeout()? I think that would fix it until we have a better fix for the queueing system.
Description of the bug/issue
When I use waitUntil in a custom command I expect the test to waitUntil evaluates to true and exits or times out, but the test runs through it as waitUntil continues to execute sometimes even after the browser session is closed due to the test completing.
It seems to be related to what I am looping on/what commands I am using inside the body of the waitUntil. I noticed the behavior when using browser.execute, but not with other methods.
Steps to reproduce
npm installfollowed bynpx nightwatch -t "nightwatch/tests/notWaiting.ts" --headlessThe test on line 14 is running dropDownPage.problemCommand(...) asserting the wrong title. "problemCommand" is a custom command that uses waitUntil to wait until the title equals the expectation. It gets the title using browser.execute inside the waitUntil body. The test is purposefully using the wrong title. For diagnostics, it is also logging the title of the page each time waitUntil loops.
You can see in the screenshot it logs the title "The Internet" twice, but then falls through to the remainder of the script and passes the test. It should have remained there and failed the test since the title will never match. The command keeps running after the browser closes causing the
Error Error while running .executeScript() protocol action: This driver instance does not have a valid session ID (did you call WebDriver.quit()?) and may no longer be used.If you comment out line 14 and uncomment the equivalent code on lines 15-21 the test will work as expected
Sample test
Command to run
Verbose Output
Nightwatch Configuration
Nightwatch.js Version
3.0.0-beta.1
Node Version
18.16.0
Browser
Chrome 113
Operating System
Windows
Additional Information
Happens on 2.6.21 and the 3.0 beta. Not sure if it is related to https://github.com/nightwatchjs/nightwatch/issues/3741