search

nightwatchjs / nightwatch-docs

Source for https://nightwatchjs.org website
https://nightwatchjs.org
MIT License
95 stars 193 forks source link

Missing documentation for running in safari with nightwatch 1.x #94

Open alex7egli opened 5 years ago

alex7egli commented 5 years ago

I cannot find any examples of nightwatch configs or clear instructions on how to run nightwatch 1.x tests against safari, even though the documentation implies that it is supported. The wiki page (https://github.com/nightwatchjs/nightwatch/wiki/Running-tests-in-Safari) says if you have Safari 10 or greater then it just works if you set browserName in your desiredCapabilities, but this does not work and it is not obvious or clear how you would run nightwatch against safari. The main documentation does not have any examples that include safari. The wiki page also includes some confusing instructions about manually opening a browser extension if you have an older version of selenium but those aren't clear either.

I've tried including selenium configuration information with the downloaded selenium jar but it did not work and the documentation says you should not use the standalone jar anymore anyway.

I've tried specifying the path to the safaridriver on my mac as the webdriver path but it just fails saying I have missing command line arguments for safaridriver.

beatfactor commented 5 years ago

Yes, we haven't had the chance yet to properly document the usage with Safari, but it is very straightforward indeed. And if you find that something is missing or can be improved in the Wiki page, feel free to update it.

You can have a look at testing with webdriver in safari and also about webdriver for safari pages from Apple, but the gist is:

Bear in mind that using Safari with Nightwatch hasn't been tested very well.

alex7egli commented 5 years ago

I originally tried that but it also does not work. I ran the following command with the following nightwatch.json and got the resulting error. It doesn't seem able to actually use the safaridriver, possibly authentication issues? When I ran /usr/bin/safaridriver --enable from the command line it prompted me to enter my password. Is there a way for me to pass my user's username and password to the nightwatch runner in the config file?

Command: nightwatch -c __tests__/e2e/nightwatch.conf.js -e safari __tests__/e2e/suites/sanity.js

nightwatch.json: NOTE: I also tried with the selenium section removed but I got the same error. The Apple docs say you need the latest selenium jar so it implies I would need that included in nightwatch as well.

{
  "src_folders": [
    "__tests__/e2e/tests"
  ],
  "output_folder": "__tests__/e2e/reports",
  "custom_commands_path": "",
  "custom_assertions_path": "",
  "page_objects_path": "",
  "globals_path": "",

  "webdriver": {
    "start_process": true,
    "server_path": "/usr/bin/safaridriver"
  },

  "selenium" : {
    "selenium" : {
      "start_process": true,
      "port": 4444,
      "host": "localhost",
      "server_path": "__tests__/e2e/selenium-server-standalone-3.141.59.jar",
      "cli_args": {
      }
    }
  },

  "test_settings": {
    "default": {
      "desiredCapabilities": {
        "browserName": "chrome",
        "javascriptEnabled": true,
        "acceptSslCerts": true,
        "chromeOptions": {
          "args": [
            "--window-size=1440,900"
          ]
        }
      }
    },
    "safari": {
      "launch_url": "localhost",
      "desiredCapabilities": {
        "browserName":"safari"
      },
      "globals": {
        "env": "dev"
      }
    }
  }
}

Error: iMac:Web dev$ npm run test:sanity:safari

web@3.0.0 test:sanity:safari /Users/dev/web nightwatch -c tests/e2e/nightwatch.conf.js -e safari tests/e2e/suites/sanity.js

Timeout while trying to connect to SafariDriver on port 4445. at Socket.socketErrorListener (_http_client.js:387:9) at emitErrorNT (internal/streams/destroy.js:64:8) at _combinedTickCallback (internal/process/next_tick.js:138:11) objc[66307]: Class AMSupportURLConnectionDelegate is implemented in both /System/Library/PrivateFrameworks/EmbeddedOSInstall.framework/Versions/A/EmbeddedOSInstall (0x7fff97ad7748) and /System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A/MobileDevice (0x10a424600). One of the two will be used. Which one is undefined. objc[66307]: Class AMSupportURLSession is implemented in both /System/Library/PrivateFrameworks/OSPersonalization.framework/Versions/A/OSPersonalization (0x7fff98e059f0) and /System/Library/PrivateFrameworks/MobileDevice.framework/Versions/A/MobileDevice (0x10a424650). One of the two will be used. Which one is undefined.

npm ERR! code ELIFECYCLE npm ERR! errno 10 npm ERR! web@3.0.0 test:sanity:safari: nightwatch -c __tests__/e2e/nightwatch.conf.js -e safari __tests__/e2e/suites/sanity.js npm ERR! Exit status 10 npm ERR! npm ERR! Failed at the web@3.0.0 test:sanity:safari script. npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in: npm ERR! /Users/dev/.npm/_logs/2019-01-14T15_04_52_898Z-debug.log

beatfactor commented 5 years ago

When I ran /usr/bin/safaridriver --enable from the command line it prompted me to enter my password. Is there a way for me to pass my user's username and password to the nightwatch runner in the config file?

No, not that I know of. You need to run /usr/bin/safaridriver --enable and enter your password to enable it, but you only have to do that once.

Regarding your config, you should set selenium.start_server = false if you have webdriver.start_server set to true. You should also set webdriver.port = 4445 when running against safari.

alex7egli commented 5 years ago

Thanks! I got it running with this config file and by opening Safari and checking Develop > Allow Remote Automation:

{
  "src_folders": [
    "__tests__/e2e/tests"
  ],
  "output_folder": "__tests__/e2e/reports",
  "custom_commands_path": "",
  "custom_assertions_path": "",
  "page_objects_path": "",
  "globals_path": "",

  "webdriver": {
    "start_process": true,
    "server_path": "/usr/bin/safaridriver",
    "port": 4445
  },

  "test_settings": {
    "default": {
      "desiredCapabilities": {
        "browserName": "safari",
        "javascriptEnabled": true,
        "acceptSslCerts": true
      }
    },
    "safari": {
      "launch_url": "localhost",
      "desiredCapabilities": {
        "browserName":"safari"
      },
      "globals": {
        "env": "dev"
      }
    }
  }
}

The tests don't all pass the way they do in Chrome but it's probably just browser quirks at this point.

beatfactor commented 5 years ago

The failures could be due to which mode is safaridriver using (try switching the use_legacy_jsonwire between true and false). When running in W3C mode (i.e. use_legacy_jsonwire = false), there might be commands which are not fully implemented by Nightwatch (check the --verbose output). If you see any protocol related issues (things not being implemented, type errors etc.), please report them as issues on the main repo.

aadityapawar commented 5 years ago

@beatfactor for --enable , does that need admin access. Any idea ?

saranyamohandas commented 5 years ago

Is there a way for standalone usage for safari as in with chrome?

saranyamohandas commented 5 years ago

Hi I am trying to setup nightwatch for safari. I am getting the below error. Any help/suggestions would be of great help and highly appreciated!

SafariDriver process exited with code: 1 safaridriver: unrecognized option `--legacy'

   at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)

safaridriver: unrecognized option `--legacy'

jensbodal commented 5 years ago

The problem appears to be the opposite now with the latest version of safari. use_legacy_jsonwire appears to be set to true by default, and latest safaridriver only "supports the protocol as defined by the W3C WebDriver specification."

Here's the config that works for me:

"safari": {
  "webdriver": {
    "use_legacy_jsonwire": false,
    "start_process": true,
    "server_path": "/usr/bin/safaridriver",
    "port": 4445
  },
  "desiredCapabilities": {
    "browserName": "safari"
  }
}
beatfactor commented 5 years ago

It's no longer set to legacy in v1.1.10.

beatfactor commented 5 years ago

See also https://github.com/nightwatchjs/nightwatch-website-tests/blob/master/nightwatch.conf.js#L33

alex7egli commented 5 years ago

jensbodal's comment fixed the issue of nightwatch not running on safari v12 anymore, but now it can't find a few commands, like the basic waitForElementVisible command. When I run it I get these errors in the console a lot:

Error while running .setWindowSize() protocol action: The command 'POST /session/AA6348AA-20F6-4570-A1ED-F1EB95161E49/window/current/size' was not found.

Error while running .setWindowPosition() protocol action: The command 'POST /session/AA6348AA-20F6-4570-A1ED-F1EB95161E49/window/current/position' was not found.

Error while running .isElementDisplayed() protocol action: The command 'GET /session/AA6348AA-20F6-4570-A1ED-F1EB95161E49/element/node-4F5C829D-8E90-43BF-812F-CB6378DF1CA4/displayed' was not found.

I looked up the issue and found old posts related to nightwatch 0.9.x to 1.x upgrade, but nothing recent. My config is now:

const chromedriver = require('chromedriver');
module.exports = {
  src_folders: [
    '__tests__/e2e/tests'
  ],
  output_folder: '__tests__/e2e/reports',
  custom_commands_path: '',
  custom_assertions_path: '',
  page_objects_path: '',
  globals_path: '',

  // Default to safari since it doesn't work if you try and specify
  // webdriver settings inside the safari setup
  webdriver: {
    start_process: true
  },

  test_settings: {
    default: {
      desiredCapabilities: {
        javascriptEnabled: true,
        acceptSslCerts: true,
        chromeOptions: {
          args: [
            '--window-size=1440,900',
            '--disable-web-security'
          ]
        }
      }
    },
    safari: {
      launch_url: 'https://host.com', // changed to fake for public post
      desiredCapabilities: {
        browserName: 'safari'
      },
      webdriver: {
        server_path: '/usr/bin/safaridriver',
        port: 4445,
        use_legacy_jsonwire: false
      },
      globals: {
        env: 'dev',
        browserName: 'safari'
      }
    }
  }
};
alex7egli commented 5 years ago

I updated nightwatch to v1.2.1, updated config as specified by jensbodal, and now the tests run in safari 12.x! Some commands are not implemented (e.g. waitForElementVisible) but the tests do start running. Thanks for all the info and help!

HuotChu commented 4 years ago

This is very easy to accomplish using the Selenium server:

        selenium: {
            start_process: true,
            server_path: require('selenium-server').path,
            port: 4445,
            cli_args: {
                'webdriver.safari.driver': '/usr/bin/safaridriver'
            }
        },
        'safari': {
            extends: 'selenium',
            desiredCapabilities: {
                browserName: 'safari'
            }
        }
jacksontwu commented 4 years ago

I have met this problem, can anyone tell me how to solve it? error: safaridriver: unrecognized option `--w3c'

my setting : safari: { desiredCapabilities: { browserName: 'safari' }, webdriver: { server_path: '/usr/bin/safaridriver', port: 4445, use_legacy_jsonwire: false }, },

leemr commented 4 years ago

@jacksonwu1992 Take a look here. https://github.com/nightwatchjs/nightwatch/blob/1cffb277475c74d16cb2078fc6489ceeac48d819/lib/runner/wd-instances/safaridriver.js#L74-L78 Just delete the --w3c or --legacy option so that those options aren't pushed. That will at least get you running. For medium term, maybe fork the project till there's a fix.