[bug] tauri-driver's native webdriver is not always synced with itself.

[chippers]

Describe the bug

Based on a Twitter thread about using WebDriver, a developer seems to have run into a timing issue while running a WebdriverIO test suite on Windows in CI. Link to the failed CI run. Based on the logs, it seems like the following likely took place:

1. WebdriverIO starts and prepares to run 2 workers 1 at a time, due to the config
2. WebdriverIO starts tauri-driver using the beforeSession function from the config
3. tauri-driver is started
4. tauri-driver starts the native WebDriver tool (msedgedriver.exe)
5. msedgedriver.exe starts (based on log output)
6. The first test's session fails to communicate with the socket, citing a socket hang up.
7. First test fails
8. tauri-driver is killed based on the afterSession function from the config
9. Second WebdriverIO session starts for the second spec file
10. Like step 2, it starts tauri-driver
11. tauri-driver is started
12. tauri-driver starts msedgedriver.exe
13. msedgedriver.exe fails to start, citing the port already being in use
14. The second test runs, successfully communicating through tauri-driver to msedgedriver.exe to run and pass the test

In short, my conclusion is that there is likely a timing issue between starting tauri-driver and it starting the native WebDriver which causes the test suite to try and communicate with it before it is ready. The second test likely passed because it used the native WebDriver started in the first test and both it and tauri-driver were ready. I cannot reproduce this locally so this is all speculation based on the CI log output and my current understanding of how the typical workflow goes for the test suite.

Reproduction

Cannot reproduce locally. See log here.

Expected behavior

Both tests should pass. The native WebDriver should not fail to start after the first one should have been killed.

Platform and versions

Operating System - Windows, version 10.0.19044 X64
Webview2 - 96.0.1054.41
Visual Studio Build Tools:
   - Visual Studio Build Tools 2022

Node.js environment
  Node.js - 16.14.0
  @tauri-apps/cli - 1.0.0-rc.5
  @tauri-apps/api - 1.0.0-rc.1

Global packages
  npm - 8.3.1
  pnpm - Not installed
  yarn - 1.22.15

Rust environment
  rustup - 1.24.3
  rustc - 1.59.0
  cargo - 1.59.0
  toolchain - stable-x86_64-pc-windows-msvc

App directory structure
/.github
/.svelte-kit
/bin
/build
/node_modules
/sample_data
/src
/src-tauri
/static
/tests

App
  tauri - 1.0.0-rc.2
  tauri-build - 1.0.0-rc.2
  tao - 0.6.2
  wry - 0.13.1
  build-type - bundle
  CSP - default-src blob: data: filesystem: ws: http: https: 'unsafe-eval' 'unsafe-inline'
  distDir - ../build
  devPath - http://127.0.0.1:3000/
  framework - Svelte

Stack trace

Run yarn test
  yarn test
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
yarn run v1.22.17
$ yarn --cwd tests/webdriverio run test
$ wdio run wdio.conf.ts

Execution of 2 workers started at 2022-02-27T22:24:45.882Z

[0-0] RUNNING in undefined - D:\a\Compotes\Compotes\tests\webdriverio\specs\home.ts
[0-0] Starting MSEdgeDriver 98.0.1108.56 (9a336a18eae89157b3c7ea0568a9cbced8ebc3f7) on port 4445
[0-0] Only local connections are allowed.
[0-0] Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping MSEdgeDriver safe.
[0-0] 2022-02-27T22:24:49.628Z ERROR webdriver: RequestError: socket hang up
[0-0]     at ClientRequest.<anonymous> (D:\a\Compotes\Compotes\tests\webdriverio\node_modules\got\dist\source\core\index.js:962:111)
[0-0]     at Object.onceWrapper (node:events:640:26)
[0-0]     at ClientRequest.emit (node:events:532:35)
[0-0]     at ClientRequest.emit (node:domain:475:12)
[0-0]     at ClientRequest.origin.emit (D:\a\Compotes\Compotes\tests\webdriverio\node_modules\@szmarczak\http-timer\dist\source\index.js:43:20)
[0-0]     at Socket.socketOnEnd (node:_http_client:466:9)
[0-0]     at Socket.emit (node:events:532:35)
[0-0]     at Socket.emit (node:domain:475:12)
[0-0]     at endReadableNT (node:internal/streams/readable:1346:12)
[0-0]     at connResetException (node:internal/errors:691:14)
[0-0]     at Socket.socketOnEnd (node:_http_client:466:23)
[0-0]     at Socket.emit (node:events:532:35)
[0-0]     at Socket.emit (node:domain:475:12)
[0-0]     at endReadableNT (node:internal/streams/readable:1346:12)
[0-0]     at processTicksAndRejections (node:internal/process/task_queues:83:21)
[0-0] 2022-02-27T22:24:49.631Z ERROR @wdio/runner: Error: Failed to create session.
[0-0] socket hang up
[0-0]     at startWebDriverSession (D:\a\Compotes\Compotes\tests\webdriverio\node_modules\webdriver\build\utils.js:72:15)
[0-0]     at processTicksAndRejections (node:internal/process/task_queues:96:5)
[0-0]     at async Function.newSession (D:\a\Compotes\Compotes\tests\webdriverio\node_modules\webdriver\build\index.js:42:45)
[0-0]     at async remote (D:\a\Compotes\Compotes\tests\webdriverio\node_modules\webdriverio\build\index.js:73:22)
[0-0]     at async Runner._startSession (D:\a\Compotes\Compotes\tests\webdriverio\node_modules\@wdio\runner\build\index.js:223:56)
[0-0]     at async Runner._initSession (D:\a\Compotes\Compotes\tests\webdriverio\node_modules\@wdio\runner\build\index.js:176:25)
[0-0]     at async Runner.run (D:\a\Compotes\Compotes\tests\webdriverio\node_modules\@wdio\runner\build\index.js:88:19)
[0-0] FAILED in undefined - D:\a\Compotes\Compotes\tests\webdriverio\specs\home.ts
[0-1] Starting MSEdgeDriver 98.0.1108.56 (9a336a18eae89157b3c7ea0568a9cbced8ebc3f7) on port 4445
[0-1] Only local connections are allowed.
[0-1] Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping MSEdgeDriver safe.
[0-1] RUNNING in undefined - D:\a\Compotes\Compotes\tests\webdriverio\specs\operations.ts
[0-1] MSEdgeDriver was started successfully.
[0-0] [1646000692.648][SEVERE]: bind() returned an error: Only one usage of each socket address (protocol/network address/port) is normally permitted. (0x2740)
[0-0] IPv6 port not available. Exiting...
[0-1] PASSED in undefined - D:\a\Compotes\Compotes\tests\webdriverio\specs\operations.ts

 "spec" Reporter:
------------------------------------------------------------------
[webview2 98.0.1108.56 windows #0-1] Running: webview2 (v98.0.1108.56) on windows
[webview2 98.0.1108.56 windows #0-1] Session ID: f4de589355b97094da10a3f5e7076bbf
[webview2 98.0.1108.56 windows #0-1]
[webview2 98.0.1108.56 windows #0-1] » \specs\operations.ts
[webview2 98.0.1108.56 windows #0-1] Operations page
[webview2 98.0.1108.56 windows #0-1]    ✓ can display operations list
[webview2 98.0.1108.56 windows #0-1]
[webview2 98.0.1108.56 windows #0-1] 1 passing (453ms)

Spec Files:  1 passed, 1 failed, 2 total (100% completed) in 00:00:12 

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Error: Process completed with exit code 1.

Additional context

I'm not sure what the solution is. The desired behavior change seems to be two things, for starters

1. Some way for tauri-driver to communicate that both itself and the child-process running the native WebDriver server is ready
2. Better syncing between tauri-driver being killed along with the child process which should be killed with it.

Being not able to reproduce it locally will make it annoying to debug and validate that these fixes addressed the underlying problem. Not sure if it can even be reliably reproduced on CI.

[Pierstoval]

Maybe tauri driver shouldn't be killed at "afterSession()" but instead after the whole test suite is finished? This way, it's okay to "sleep" for 1 second (at most) to let tauri-driver start, and then run the whole suite, and the driver is only killed once afterwards.

WDYT?

[Kumassy]

I hit the same issue. The test works on my local Windows 10 PC, but not work on Github hosted windows-latest VM.