@ui5/webcomponents-tools reliance on Chrome Browser installation

[codefactor]

Describe the bug SuccessFactors is trying to build our own library of UI5 web components. I have followed the tutorial to create a new project, which is a package of the common-components github (uses Lerna).

I cannot run the "npm run test" command locally - this same issue is happening on all SuccessFactors Dev VM machine. I suspect that your team is able to get the test to run because of some other type of global NPM setting or binary installation of some kind. This is blocking the merge of the code to master, so I am currently working on a branch waiting for this issue to be resolved and then the pre-commit checks on the PR can pass.

To reproduce

Using a machine which does not have a local Chrome installation, and/or does not have general internet connectivity outside the SAP Corporate network:

git clone https://github.wdf.sap.corp/xweb/common-components
git co feature/header
cd common-components/packages/webcomponents
npm i

Run a test and see if it works

npm test

Isolated example Please provide an isolated example if possibile, by forking CodeSandbox

Expected behavior

1. The test should pass on any machine regardless of your local installations as long as you have general internet access (the build fails right now on XMAKE due to restricted internet access, but we are working on this separately)
2. Changing a file and waiting for the build and refreshing the browser should work consistently without need to Ctrl+c the server and start again

Screenshots If applicable, add screenshots to help explain your problem.

Context

• UI5 Web Components version: latest next version (5/16/2020)
• OS/Platform: Ubuntu 16.04.3 LTS
• Browser (if relevant): {...}
• Other information: v10.16.13 of node, v6.0.0 of npm

Affected components (if known)

@ui5/webcomponents-tools

Log output / Any errors in the console

> @xweb/common-webcomponents@1.0.0-rc.14 test /data/sfsf/workspace/trunk/nogradle/common-components/packages/webcomponents
> wc-dev test

nps is executing `test` : concurrently "nps serve" "nps test.run" --kill-others --success first
[0] nps is executing `serve` : nps serve.prepare serve.run
[1] nps is executing `test.run` : cross-env WDIO_LOG_LEVEL=error FORCE_COLOR=0 wdio config/wdio.conf.js
[0] nps is executing `serve.prepare` : copy-and-watch "/data/sfsf/workspace/trunk/nogradle/common-components/packages/webcomponents/node_modules/@ui5/webcomponents-tools/components-package/serve.json" dist/
[0] [COPY] /data/sfsf/workspace/trunk/nogradle/common-components/packages/webcomponents/node_modules/@ui5/webcomponents-tools/components-package/serve.json to dist/serve.json
[0] nps is executing `serve.run` : serve --no-clipboard -l 8081 dist
[1] 
[1] Execution of 1 spec files started at 2020-05-16T15:34:52.557Z
[1] 
[1] (node:53607) ExperimentalWarning: The dns.promises API is experimental
[1] (node:53607) ExperimentalWarning: The fs.promises API is experimental
[1] Starting ChromeDriver 81.0.4044.69 (6813546031a4bc83f717a2ef7cd4ac6ec1199132-refs/branch-heads/4044@{#776}) on port 9515
[1] Only local connections are allowed.
[1] Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
[0] WARNING: Checking for updates failed (use `--debug` to see full error)
[0] INFO: Discovered configuration in `serve.json`
[0] INFO: Accepting connections at http://localhost:8081
[1] [0-0] (node:53642) ExperimentalWarning: The dns.promises API is experimental
[1] [0-0] (node:53642) ExperimentalWarning: The fs.promises API is experimental
[1] [0-0] RUNNING in chrome - /test/specs/Header.spec.js
[1] [0-0] 2020-05-16T15:34:54.836Z ERROR webdriver: Request failed with status 500 due to session not created: session not created: This version of ChromeDriver only supports Chrome version 81
[1] [0-0] 2020-05-16T15:34:54.837Z ERROR webdriver: session not created: session not created: This version of ChromeDriver only supports Chrome version 81
[1]     at getErrorFromResponseBody (/data/sfsf/workspace/trunk/nogradle/common-components/packages/webcomponents/node_modules/webdriver/build/utils.js:121:10)
[1]     at WebDriverRequest._request (/data/sfsf/workspace/trunk/nogradle/common-components/packages/webcomponents/node_modules/webdriver/build/request.js:148:56)
[1]     at process._tickCallback (internal/process/next_tick.js:68:7)
[1] 2020-05-16T15:34:54.838Z ERROR @wdio/runner: Error: Failed to create session.
[1] session not created: This version of ChromeDriver only supports Chrome version 81
[1]     at startWebDriverSession (/data/sfsf/workspace/trunk/nogradle/common-components/packages/webcomponents/node_modules/webdriver/build/utils.js:45:11)
[1]     at process._tickCallback (internal/process/next_tick.js:68:7)
[1] [0-0]  Error:  Failed to create session.
[1] session not created: This version of ChromeDriver only supports Chrome version 81
[1] [0-0] FAILED in chrome - /test/specs/Header.spec.js
[1] 
[1] Spec Files:  0 passed, 1 failed, 1 total (100% completed) in 00:00:02 
[1] 
[1] The script called "test.run" which runs "cross-env WDIO_LOG_LEVEL=error FORCE_COLOR=0 wdio config/wdio.conf.js" failed with exit code 1 https://github.com/sezna/nps/blob/master/other/ERRORS_AND_WARNINGS.md#failed-with-exit-code
[1] nps test.run exited with code 1
--> Sending SIGTERM to other processes..
[0] 
[0] INFO: Gracefully shutting down. Please wait...
[0] nps serve exited with code SIGTERM

Checking the chrome driver from npm -- note that I have already deleted the node_modules folder locally and reinstalled a few times.

 $  npm ls chromedriver
@xweb/common-webcomponents@1.0.0-rc.14 /data/sfsf/workspace/trunk/nogradle/common-components/packages/webcomponents
└─┬ @ui5/webcomponents-tools@0.0.0-9195c41b9
  └── chromedriver@81.0.0 

[codefactor]

@vladitasev ,

On this issue I guess right now we will do more investigation from our side to see what we can do to change the way the tests are executed. We may just change it entirely and use our existing karma-based testing which, if I'm not mistaken, uses it's own Chrome binary installation which I think comes with Puppeteer.

Unfortunately, we rely on the package to be delivering a more complete installation and one which can be done purely on the SAP corporate network - rather than depending on the developer and/or OPs to install any binaries (such as Chrome or global install the Chrome Driver). Also, if I'm not mistaken we use Chromium, for some reason - maybe because this one is more approved or already readily available?

Are you aware of any configuration or hooks we can do to update the way the tools interacts with a Browser to run it's tests?

[vladitasev]

@codefactor ,

For general chromedriver settings, see: https://www.npmjs.com/package/chromedriver

I believe it would be easiest to automate chromedriver download from nexus, f.e. with curl in a pre-build task and then point to the downloaded binary, f.e. taken from the page above:

npm install chromedriver --chromedriver_filepath=/path/to/chromedriver_mac64.zip

You can also use .npmrc or PATH, all methods are described there in the "Custom binaries files".

[vladitasev]

Now it is possible to manage ChromeDriver separately: https://github.com/SAP/ui5-webcomponents/issues/1682

For details, see: https://github.com/SAP/ui5-webcomponents/blob/master/docs/dev/Testing%20Web%20Components.md