search

nightwatchjs / nightwatch

Integrated end-to-end testing framework written in Node.js and using W3C Webdriver API. Developed at @browserstack
https://nightwatchjs.org
MIT License
11.79k stars 1.31k forks source link

Browserstack works on 1.0.14 but not 1.0.16 #1943

Closed gregoryduckworth closed 5 years ago

gregoryduckworth commented 5 years ago

When attempting to run the tests against Browserstack (using version 1.0.16) the console throws the error Response 500 POST http://hub-cloud.browserstack.com/session, however when using 1.0.14 the tests are run fine against Browserstack.

The only thing that has changed is the version number of Nightwatch as my configuration files have stayed the same.

beatfactor commented 5 years ago

Can you post your config and verbose log please?

phsultan commented 5 years ago

Hi,

Mine don't work with BrowserStack, and neither locally (standalone selenium server running version 3.14.0). Seems like 1.0.16 sends POST to /session whereas 1.0.14 sends POST to /wd/hub/session.

Hope the details below will help solve the issue.

Command:

./node_modules/nightwatch/bin/nightwatch --test sites/www.google.com.js --env chrome --verbose
[Sites/Www Google Com] Test Suite
=================================
   Request POST  /session  
   { desiredCapabilities: 
      { browserName: 'chrome',
        acceptSslCerts: true,
        platform: 'ANY',
        javascriptEnabled: true,
        nativeEvents: true,
        chromeOptions: 
         { args: 
            [ 'headless',
              'no-sandbox',
              'allow-file-access-from-files',
              'use-fake-device-for-media-stream',
              'use-fake-ui-for-media-stream',
              'disable-translate',
              'no-process-singleton-dialog',
              'mute-audio' ] },
        name: 'Sites/Www Google Com' } }
   Response 404 POST /session (13ms)
   { status: -1,
     value: '<!DOCTYPE html>\n<html lang="en">\n<head>\n  <meta charset="UTF-8">\n  <link rel="stylesheet" type="text/css" href="/assets/displayhelpservlet.css" media="all"/>\n  <link href="/assets/favicon.ico" rel="icon" type="image/x-icon" />\n  <script src="/assets/jquery-3.1.1.min.js" type="text/javascript"></script>\n  <script src="/assets/displayhelpservlet.js" type="text/javascript"></script>\n  <script type="text/javascript">\n    var json = Object.freeze(\'{"consoleLink": "\\u002fwd\\u002fhub","type": "Standalone","class": "org.openqa.grid.web.servlet.DisplayHelpServlet$DisplayHelpServletConfig","version": "3.14.0"}\');\n  </script>\n</head>\n<body>\n\n<div id="content">\n  <div id="help-heading">\n    <h1><span id="logo"></span></h1>\n    <h2>Selenium <span class="se-type"></span>&nbsp;v.<span class="se-version"></span></h2>\n  </div>\n\n  <div id="content-body">\n    <p>\n      Whoops! The URL specified routes to this help page.\n    </p>\n    <p>\n      For more information about Selenium <span class="se-type"></span> please see the\n      <a class="se-docs">docs</a> and/or visit the <a class="se-wiki">wiki</a>.\n      <span id="console-item">\n        Or perhaps you are looking for the Selenium <span class="se-type"></span> <a class="se-console">console</a>.\n      </span>\n    </p>\n    <p>\n      Happy Testing!\n    </p>\n  </div>\n\n  <div>\n    <footer id="help-footer">\n      Selenium is made possible through the efforts of our open source community, contributions from\n      these <a href="https://github.com/SeleniumHQ/selenium/blob/master/AUTHORS">people</a>, and our\n      <a href="http://www.seleniumhq.org/sponsors/">sponsors</a>.\n   </footer>\n  </div>\n </div>\n\n</body>\n</html>' }
   An occurred error while retrieving a new session

_________________________________________________

TEST FAILURE: 1 error during execution 0 tests failed, 0 passed. 44ms

 ✖ sites/www.google.com

  An occurred error while retrieving a new session

   SKIPPED:
   - Google's Search Functionality

 Wrote report file to: reports/sites/www.google.com.xml.

nightwatch.json:

{
  "src_folders": [
    "sites"
  ],
  "output_folder": "reports",
  "selenium": {
    "start_process": false
  },
  "test_settings": {
    "safari": {
      "desiredCapabilities": {
        "browserName": "safari",
        "javascriptEnabled": true,
        "acceptSslCerts": true
      }
    },
    "chrome": {
      "desiredCapabilities": {
        "browserName": "chrome",
        "javascriptEnabled": true,
        "acceptSslCerts": true,
        "nativeEvents": true,
        "chromeOptions": {
          "args": [
            "headless",
            "no-sandbox",
            "allow-file-access-from-files",
            "use-fake-device-for-media-stream",
            "use-fake-ui-for-media-stream",
            "disable-translate",
            "no-process-singleton-dialog",
            "mute-audio"
          ]
        }
      }
    },
    "firefox": {
      "desiredCapabilities": {
        "browserName": "firefox",
        "javascriptEnabled": true,
        "acceptSslCerts": true,
        "nativeEvents": true,
        "moz:firefoxOptions": {
          "log": {
            "level": "trace"
          },
          "args": [
            "-headless",
            "-no-remote"
          ],
          "prefs": {
            "browser.cache.disk.enable": false,
            "browser.cache.disk.capacity": 0,
            "browser.cache.disk.smart_size.enabled": false,
            "browser.cache.disk.smart_size.first_run": false,
            "browser.sessionstore.resume_from_crash": false,
            "browser.startup.page": 0,
            "media.navigator.streams.fake": true,
            "media.navigator.permission.disabled": true,
            "device.storage.enabled": false,
            "media.gstreamer.enabled": false,
            "browser.startup.homepage": "about:blank",
            "browser.startup.firstrunSkipsHomepage": false,
            "extensions.update.enabled": false,
            "app.update.enabled": false,
            "network.http.use-cache": false,
            "browser.shell.checkDefaultBrowser": false
          }
        }
      }
    }
  }
}

sites/www.google.com.js:

module.exports = {
  'Google\'s Search Functionality' : function (browser) {
    browser
      .url('https://www.google.com/ncr')
      .waitForElementVisible('body', 5000)
      .setValue('input[type=text]', 'BrowserStack\n')
      .pause(10000)
      .assert.title('BrowserStack - Google Search')
      .end();
  }
};

Command for selenium server:

java -Dwebdriver.chrome.driver='./node_modules/chromedriver/lib/chromedriver/chromedriver' -Dwebdriver.gecko.driver='./node_modules/geckodriver/geckodriver' -jar ./node_modules/selium-server/lib/runner/selenium-server-standalone-3.14.0.jar -role standalone --debug
aberonni commented 5 years ago

@gregoryduckworth @phsultan I think this can be fixed by making a small change in your configuration: https://github.com/nightwatchjs/nightwatch/wiki/Migrating-to-Nightwatch-1.0#configuration

phsultan commented 5 years ago

@aberonni the proposed workaround fixed it in my case, thanks!

gregoryduckworth commented 5 years ago

Thanks @aberonni, using 

webdriver: {
    ...
    default_path_prefix: '/wd/hub',
  },

solved the issue for me!

beatfactor commented 5 years ago

@gregoryduckworth it would still be interesting to see your config which was causing the failure.

beatfactor commented 5 years ago

@phsultan Could you post your browserstack config as well?

phsultan commented 5 years ago

Contrary to what I initially posted, BrowserStack works ok in my case, and the test command fails locally only.

Configuration file:

$ cat bs_nightwatchconfig.js 
nightwatch_config = {
  src_folders: [
    'sites'
  ],
  selenium : {
    "start_process" : false,
    "host" : "hub-cloud.browserstack.com",
    "port" : 80
  },
  common_capabilities: {
    'browserstack.user': 'xxxxxxxxxxxxxxxxx',
    'browserstack.key': 'xxxxxxxxxxxxxxxxx',
    'acceptSslCerts': true
  },
  test_settings: {
   default: {},
   chrome: {
     desiredCapabilities: {
       browserName: "chrome",
       os: 'OS X',
       os_version: 'Mojave',
       browser: 'Chrome',
       browser_version: '70.0',
       chromeOptions: {
         args: [
           "use-fake-device-for-media-stream",
           "use-fake-ui-for-media-stream",
         ]
       }
     }
   },
   firefox: {
     desiredCapabilities: {
       browserName: "firefox",
       os: 'OS X',
       os_version: 'Mojave',
       browser: 'Firefox',
       browser_version: '63.0',
       "moz:firefoxOptions": {
          prefs: {
            "media.navigator.streams.fake": true,
            "media.navigator.permission.disabled": true,
          }
        }
     }
   },
   safari: {
     desiredCapabilities: {
       browserName: "safari",
       javascriptEnabled : true,
       acceptSslCerts : true,
       nativeEvents: true,
       real_mobile: false,
       'browserstack.debug': true,
       'browserstack.use_w3c': false,
       'browserstack.selenium_version' : '3.14.0',
       os: 'OS X',
       os_version: 'Mojave',
       browser: 'Safari',
       browser_version: '12.0',
     }
   },
   iOS: {
     desiredCapabilities: {
       'os_version' : '12.1',
       'device' : 'iPhone 8',
       'real_mobile' : 'true',
       'browserstack.console': 'errors',
     }
   },
   android: {
     desiredCapabilities: {
       browserName: "chrome",
       'os_version' : '7.0',
       'device' : 'Samsung Galaxy S8',
       'real_mobile' : 'true',
       'browserstack.console': 'errors',
       chromeOptions: {
         args: [
           "use-fake-device-for-media-stream",
           "use-fake-ui-for-media-stream",
         ]
       }
     }
   }
  }
};

// Code to support common capabilites
for(var i in nightwatch_config.test_settings){
  var config = nightwatch_config.test_settings[i];
  config['selenium_host'] = nightwatch_config.selenium.host;
  config['selenium_port'] = nightwatch_config.selenium.port;
  config['desiredCapabilities'] = config['desiredCapabilities'] || {};
  for(var j in nightwatch_config.common_capabilities){
    config['desiredCapabilities'][j] = config['desiredCapabilities'][j] || nightwatch_config.common_capabilities[j];
  }
}

module.exports = nightwatch_config;

This command works:

$ ./node_modules/nightwatch/bin/nightwatch --config bs_nightwatchconfig.js --test sites/www.google.com.js --env chrome --verbose

This command does not work:

$ ./node_modules/nightwatch/bin/nightwatch --test sites/www.google.com.js --env chrome --verbose

Hope this will help you debug that issue, thanks again for your help!

beatfactor commented 5 years ago

@phsultan you can also just define the common capabilities on the default test settings, and all the other sections will also have them automatically:

test_settings: {
   default: {
     desiredCapabilities: {
       'browserstack.user': 'xxxxxxxxxxxxxxxxx',
       'browserstack.key': 'xxxxxxxxxxxxxxxxx',
        acceptSslCerts: true
      }
   }
},
phsultan commented 5 years ago

As per https://github.com/nightwatchjs/nightwatch/issues/1945#issuecomment-448252723, lauching Selenium from Nightwatch fixes this issue for me. Thank you all for the help, keep up the good work!

gregoryduckworth commented 5 years ago

My config as requested (before adding the default_path_prefix):

var nightwatch_config = {
  src_folders: ['tests'],
  page_objects_path: ['pages'],
  globals_path: 'global.js',

  webdriver: {
    start_process: false,
    host: 'hub-cloud.browserstack.com',
    port: 80,
  },

  test_workers: true,

  test_settings: {
    default: {
      screenshots: {
        enabled: true,
        on_failure: true,
        on_error: true,
        path: 'screenshots/failures',
      },
      javascriptEnabled: true,
      skip_testcases_on_fail: false,
      desiredCapabilities: {
        build: 'nightwatchjs',
        project: 'Nightwatch',
        'browserstack.user': 'BROWSERSTACK_USERNAME',
        'browserstack.key': 'BROWSERSTACK_ACCESS_KEY',
        'browserstack.debug': true,
      },
    },
  },
};
beatfactor commented 5 years ago

Ok, thanks. I will update the upgrade guide. When using a remote selenium server (or when selenium server is not managed by nightwatch), you can either set the default_path_prefix as mention before or set the selenium_host / selenium_port properties, as show below:

"test_settings" : {
    "default" : {
      "selenium_host": "hub-cloud.browserstack.com",
      "selenium_port": 80
    }
  }

See also http://nightwatchjs.org/gettingstarted#test-settings.

shanehu13 commented 5 years ago

I still get the error info after I set the default_path_prefix in the webdriver in the nightwatch.conf.js Response 500 DELETE http://hub-cloud.browserstack.com/wd/hub/session (659ms) { value: { message: 'Session not started or terminated', error: [] }, sessionId: '', status: 13 } Error while running .sessionAction() protocol action: Session not started or terminated