nightwatchjs / nightwatch

Integrated end-to-end testing framework written in Node.js and using W3C Webdriver API. Developed at @browserstack
https://nightwatchjs.org
MIT License
11.83k stars 1.32k forks source link

Tests hang when using headless in environment args vs using --headless in CLI #3913

Closed reallymello closed 1 year ago

reallymello commented 1 year ago

Description of the bug/issue

When I use a chrome environment profile that has --headless in the args collection I expect the test to run in headless mode the same as if I were to use the default environment and pass --headless as a CLI argument, but I notice the tests hang in the former, especially around multiple window scenarios.

image

Steps to reproduce

Assuming your nightwatch.conf.js has the default environment profiles for chrome

chrome: {
      desiredCapabilities: {
        browserName: 'chrome',
        'goog:chromeOptions': {
          // More info on Chromedriver: https://sites.google.com/a/chromium.org/chromedriver/
          //
          // w3c:false tells Chromedriver to run using the legacy JSONWire protocol (not required in Chrome 78)
          w3c: true,
          args: [
            //'--no-sandbox',
            //'--ignore-certificate-errors',
            //'--allow-insecure-localhost',
            //'--headless'
          ],
        },
      },

uncomment all the args except --allow-insecure-localhost

 args: [
            '--no-sandbox',
            '--ignore-certificate-errors',
            //'--allow-insecure-localhost',
            '--headless'
          ],

Run the test npx nightwatch -e chrome. The test seems to hang in the script around code where a new window is launched, handles are grabbed, then the new handle is switched to.

It seems like getCurrentUrl is what hangs and eventually times out.

The same test, run using npx nightwatch --headless works without issue. I am not sure why headless when used in the chrome environment behaves differently than chrome with the --headless CLI argument.

Sample test

policyDetailsPage.click('@downloadAttachmentButton');

    browser.pause(500); // wait for window pop
    let windowHandles = await browser.window.getAllHandles();
    browser.window.switchTo(windowHandles[1]);

    const currentUrl = await browser.getCurrentUrl();

    if (!currentUrl.startsWith('blob:')) {
      browser.expect
        .title()
        .to.contain(
          'Example.PDF'
        )
        .before(10000);
    }

    windowHandles = await browser.window.getAllHandles();
    browser.window.switchTo(windowHandles[0]);
    await policyDetailsPage.click('@deleteAttachmentButton');
    await policyDetailsPage
      .waitForElementVisible('@yesConfirmButton')
      .click('@yesConfirmButton');

Command to run

npx nightwatch -e chrome

Verbose Output

npx nightwatch nightwatch\tests\uploadTest.ts  --reporter my-custom-reporter -e chrome --verbose

[Feature\ Smoke\upload] Test Suite
───────────────────────────────────────────────────────────────────────────────
⠋ Starting ChromeDriver on port 9515...
⠙ Starting ChromeDriver on port 9515...
   Request POST /session  
   {
     capabilities: {
       firstMatch: [ {} ],
       alwaysMatch: {
         browserName: 'chrome',
         'goog:chromeOptions': {
           prefs: { 'autofill.profile_enabled': false },
           w3c: true,
           args: [ '--no-sandbox', '--ignore-certificate-errors', '--headless' ]
         }
       }
     }
⠸ Starting ChromeDriver on port 9515...

[ parts omitted for brevity ]

→ Running command: click ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout...})
   Request POST /session/e498106b3555c2884570fbee35e8a785/elements  
   {
     using: 'css selector',
     value: 'button[actionref="downloadAttachment"]'
  }
   Response 200 POST /session/e498106b3555c2884570fbee35e8a785/elements (16ms)
   {
     value: [
       {
         'element-6066-11e4-a52e-4f735466cecf': '2C27578B10148B8D8290AA1BBAD917D0_element_162'
       }
     ]
  }
   Request POST /session/e498106b3555c2884570fbee35e8a785/element/2C27578B10148B8D8290AA1BBAD917D0_element_162/click  
{}
   Response 200 POST /session/e498106b3555c2884570fbee35e8a785/element/2C27578B10148B8D8290AA1BBAD917D0_element_162/click (49ms)
   { value: null }
 → Completed command: click ({name, __index, __selector, locateStrategy, pseudoSelector, parent, resolvedElement, abortOnFailure, suppressNotFoundErrors, timeout...}) (70ms)

 → Running command: pause (500)
 → Completed command: pause (500) (507ms)

 → Running command: window.getAllHandles ()
   Request GET /session/e498106b3555c2884570fbee35e8a785/window/handles  

   Response 200 GET /session/e498106b3555c2884570fbee35e8a785/window/handles (7ms)
   {
     value: [
       '84E7FAB070CDF814B5856399C4E3CD34',
       '37CC7FC50626BD6FF17157E26526DFB0',
       '2D2D2745B52B8685A7BB9137C74F0331'
     ]
  }
 → Completed command: window.getAllHandles () (8ms)

 → Running command: window.switchTo ('37CC7FC50626BD6FF17157E26526DFB0')
   Request POST /session/e498106b3555c2884570fbee35e8a785/window  
   {
     name: '37CC7FC50626BD6FF17157E26526DFB0',
     handle: '37CC7FC50626BD6FF17157E26526DFB0'
  }
   Response 200 POST /session/e498106b3555c2884570fbee35e8a785/window (4ms)
   { value: null }
 → Completed command: window.switchTo ('37CC7FC50626BD6FF17157E26526DFB0') (5ms)

 → Running command: getCurrentUrl ()
   Request GET /session/e498106b3555c2884570fbee35e8a785/url  

  Error
      GET  /session/e498106b3555c2884570fbee35e8a785/url - ECONNRESET
Error: socket hang up
    at Socket.socketCloseListener (node:_http_client:475:25)
    at Socket.emit (node:events:525:35)
    at Socket.emit (node:domain:489:12)
    at TCP.<anonymous> (node:net:322:12)
   Request GET /session/e498106b3555c2884570fbee35e8a785/url  (retry 1)

Nightwatch Configuration

chrome: {
      desiredCapabilities: {
        browserName: 'chrome',
        'goog:chromeOptions': {
          // More info on Chromedriver: https://sites.google.com/a/chromium.org/chromedriver/
          //
          // w3c:false tells Chromedriver to run using the legacy JSONWire protocol (not required in Chrome 78)
          w3c: true,
          args: [
            '--no-sandbox',
            '--ignore-certificate-errors',
            //'--allow-insecure-localhost',
            '--headless'
          ],
        },
      },

Nightwatch.js Version

3.1.3

Node Version

18.5.0

Browser

116.0.5845

Operating System

Windows 11

Additional Information

No response

AutomatedTester commented 1 year ago

This is because Google have changed the way headless works. we have sorted that internally because we defer things to Selenium but we need to update our template to remove that.

reallymello commented 1 year ago

Thanks! The world makes sense again.