Tutorial in Readme doesn't work

[toshovski]

I tried out the provided tutorial, but it tries to connect to localhost:4444. It doesn't use the selenium server hub. As workaround I used url like this:

module.exports = {
  verbose: true,
  testTimeout: 5 * 60 * 1000,
  plugins: {
    sauce: { disabled: true },
    browserstack: {
      username: 'username',
      accessKey: 'key',
      browsers: [{
        os: "Windows",
        os_version: "10",
        browser: "Chrome",
        browser_version: "60.0",
        resolution: "1024x768"
      }],
      defaults: {
        project: 'my-project',
        video: false,
        url: "http://username:key@hub.browserstack.com/wd/hub"
        }
    }
  }
};

The only problem is, that the tests stop after a timeout. If the timeout is too small, then the tests are stopped due to the timeout. If the timeout is bigger, then the tests finish and I get an browser timeout from browserstack.

[hotforfeature]

Are you using web-component-tester or my forked version web-component-tester-custom-runner? This plugin is a proof of concept that shows what can be done if BrowserRunner is abstracted and decoupled from WebDriver. It will not work with Polymer's official web-component-tester.

At the moment, you are using Browserstack's WebDriver API instead of their REST JS Testing API. The difference is that the JS Testing API offers a few more device testing features than their WebDriver API does.

If you want to use WD, you don't need this plugin at all. You can just use Browserstack's WD url as the URL for your browsers.

[toshovski]

I followed the tutorial as it is and installed the forked version of the web-component-tester-custom-runner as described in the README file. Do I need to do something else? The plugin also using the url creates the tunnel, which is also helpful.The only thing now missing is the timeout..

[hotforfeature]

You need to modify your wct.conf.js and remove the url property to use the JS API instead of WebDriver. You have to choose one or the other. If you want to use WebDriver, you shouldn't use this plugin at all.

Try installing the custom runner and plugin locally in your project instead of globally.

npm install --save-dev web-component-tester-custom-runner
npm install --save-dev wct-browserstack

Then create an npm script to run the test

package.json

{
  "name": "my-element",
  "scripts": {
    "test": "wct --plugin browserstack"
  }
}

This will ensure when you run npm test, you are using the custom runner. I suspect that if you installed globally you may still be using the Polymer wct and not the new one.

Ensure your config file looks similar to this

wct.conf.js

module.exports = {
  plugins: {
    local: { disabled: true }, // try adding this if you're struggling with the local browsers running when you don't want them to
    sauce: { disabled: true },
    browserstack: {
      username: 'myusername',
      accessKey: 'myaccesskey',
      browsers: [{
        browser: 'chrome',
        browser_version: 'latest',
        os: 'windows',
        os_version: '10'
      }]
    }
  }
};

[toshovski]

It still tries to connect to localhost:4444. Here is the configuration in my project:

package.json

{
 ....
  "scripts": {
    "test": "wct --plugin browserstack"
  },
  "dependencies": {
    "eslint": "^4.0.0",
    "wct-browserstack": "^0.2.1",
    "web-component-tester-custom-runner": "^6.1.0"
  }
}

wct.conf.js

module.exports = {
  verbose: true,
  plugins: {
    local: { disabled: true},
    sauce: { disabled: true },
    browserstack: {
      disabled: true,
      username: "myusername",
      accessKey: "myaccessKey",
      browsers: [{
        os: "Windows",
        os_version: "10",
        browser: "Chrome",
        browser_version: "60.0",
        resolution: "1024x768"
      }],
      defaults: {
        project: 'my-project',
        video: false,
      }
    }
  }
};

Here is the log from the console:

step: loadPlugins
step: configure
hook: configure
configuration: { suites: [ 'test/index.html' ],
  verbose: true,
  quiet: false,
  expanded: false,
  testTimeout: 90000,
  persistent: false,
  extraScripts: [],
  clientOptions: { root: '/components/', verbose: true },
  compile: 'auto',
  activeBrowsers: 
   [ { project: 'my-project',
       video: false,
       browser_version: '60.0',
       name: 'WCT test',
       build: undefined,
       os: 'Windows',
       os_version: '10',
       browser: 'Chrome',
       resolution: '1024x768',
       runnerCtor: [Function: BrowserstackBrowserRunner],
       wct: [Object],
       browserName: 'Chrome',
       version: '60.0',
       platform: 'Windows' } ],
  browserOptions: {},
  plugins: 
   { local: { disabled: true },
     sauce: { disabled: true },
     browserstack: 
      { disabled: false,
        username: 'username',
        accessKey: 'password',
        browsers: [Object],
        defaults: [Object],
        client: [Object] } },
  registerHooks: [Function: registerHooks],
  enforceJsonConf: false,
  webserver: 
   { hostname: 'localhost',
     _generatedIndexContent: '<!doctype html>\n<html>\n  <head>\n    <meta charset="utf-8">\n    <script>WCT = {"root":"/components/","verbose":true};</script>\n    <script>window.__generatedByWct = true;</script>\n    <script src="../web-component-tester/browser.js"></script>\n\n    <script src="../web-component-tester/data/a11ySuite.js"></script>\n</head>\n  <body>\n    <script>\n      WCT.loadSuites(["test/index.html"]);\n    </script>\n  </body>\n</html>\n' },
  root: '/home/aleksandar/Projects/web-ui/project',
  ttyOutput: true,
  _: [],
  origSuites: [ 'test/' ] }
hook: prepare
hook: prepare:browserstack-tunnel
browserstack: Creating tunnel { key: '*********************',
  localIdentifier: 'wct0.0030506852311769084',
  verbose: true }
browserstack: Tunnel established wct0.0030506852311769084
hook: prepare:webserver
step: runTests
Windows Chrome 60.0      CALL init({"project":"my-project","video":false,"browser_version":"60.0","name":"WCT test","os":"Windows","os_version":"10","browser":"Chrome","resolution":"1024x768","wct":{"client":{"server":{"host":"api.browserstack.com"},"username":"username","password":"password","authHeader":"Basic********************************"}},"browserName":"Chrome","version":"60.0","platform":"Windows","browserstack.localIdentifier":"wct0.0030506852311769084","browserstack.local":true,"variant":""})
Windows Chrome 60.0      POST /session {"desiredCapabilities":{"project":"my-project","video":false,"browser_version":"60.0","name":"WCT test","os":"Windows","os_version":"10","browser":"Chrome","resolution":"1024x768","wct":{"client":{"server":{"host":"api.browserstack.com"},"username":"username","password":"password","authHeader":"Basic ********************************"}},"browserName":"Chrome","version":"60.0","platform":"Windows","browserstack.localIdentifier":"wct0.0030506852311769084","browserstack.local":true,"variant":"","javascriptEnabled":true}}
Windows Chrome 60.0      Error code ECONNREFUSED: Unexpected error. { Error: connect ECONNREFUSED 127.0.0.1:4444
    at Object.exports._errnoException (util.js:1020:11)
    at exports._exceptionWithHostPort (util.js:1043:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1086:14)
  code: 'ECONNREFUSED',
  errno: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 4444 }

[hotforfeature]

Ok, I see the problem more clearly now. It's not that it's trying to launch the wct-local plugin, it's that it can't connect to your browserstack local tunnel. Browserstack will host the code on your localhost, which browserstack will connect to through the tunnel.

The line at exports._exceptionWithHostPort (util.js:1043:20) makes me think port 4444 is unavailable. First, check and make sure you don't have an service running on port 4444 already. I don't think this is the problem, since the tunnel should have errored out at the configuration or chosen another port, but it's a good place to start.

My next logical place to start would be your computer and network firewalls. You'll want to check and see if anything is blocking traffic to port 4444. Are you behind a proxy? You may need to add additional proxy configuration to the tunnel.