[🚫 Bug] Chromedriver version supported is only 114 and not any latest

[AshwiniJRM]

Describe the bug

Latest chrome browser available is 119, but extest get-chromedriver always pulls up chromedriver that is of version 114. Though I have tried to add chromedriver version as 119, still no luck.

Steps to reproduce

I am using the command "npm run transpile-tests-to-commonjs && rm -rf test-res && rm -rf test-resource && rm -rf ./tests/webdriver/_fixtures/tmp* && extest install-vsix --vsix_file ./test-resources/**.vsix && extest get-chromedriver && extest run-tests --mocha_config ./.mocharc.json out/tests/webdriver/tests/**/*.js" where the get-chromedriver always tries to download 114, no matter even if I have a chrome-driver version of 119 gets ignored.

Is the chrome-driver version being controlled from elsewhere? Is there a possibility to update the chromedriver version to the latest one as supported by the chrome browser. If so, which file controls this?

Logs

session not created: This version of ChromeDriver only supports Chrome version 114 Current browser version is 119.0.6045.105 with binary path /usr/bin/google-chrome

Current browser version is 119.0.6045.105 with binary path /usr/bin/google-chrome
    at Object.throwDecodedError (node_modules/selenium-webdriver/lib/error.js:524:15)
    at parseHttpResponse (node_modules/selenium-webdriver/lib/http.js:587:13)
    at Executor.execute (node_modules/selenium-webdriver/lib/http.js:515:28)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

Operating System

mac Ventura 13.4

Visual Studio Code

1.79.1

vscode-extension-tester

5.5.2

NodeJS

18.16.1

npm

9.5.1

[djelinek]

hello, thank you for reporting the issue!

I am not sure I am following.. could you please a bit more describe your usecase?

currently the vscode-extension-tester is following the latest VS Code stable release chromedriver compatible version and that one will be downloaded (or any other previous versions you will define).. but there is not option to specify custom higher version then latest vscode release supports

[djelinek]

from snippet you have provided you are trying to run tests but you are not getting VS Code binary? is that on purpose or have just missed that step?

you have also filled Visual Studio Code 1.79.1 for environment you having this issue? if that is true you need to specify that version when using extester CLI commands, that will lead to getting also appropriate chromedriver to VS Code 1.79.1 which will be for sure lower then 114.x.x.x

[AshwiniJRM]

Thanks for the quick response!! Even when extest get-vscode is present in the command (which downloads 1.84 stable version), I see that the version of chromedriver getting downloaded is still 114. command used: npm run transpile-tests-to-commonjs && rm -rf test-res && rm -rf test-resource && rm -rf ./tests/webdriver/_fixtures/tmp && extest get-vscode && extest get-chromedriver && extest install-vsix --vsix_file ./test-resources/.vsix && extest run-tests --mocha_config ./.mocharc.json out/tests/webdriver/tests//.js

logs seen: Successfully compiled 27 files with Babel (1194ms). Downloading VSCode: 1.84.2 / stable VS Code exists in local cache, skipping download Downloading ChromeDriver 114.0.5735.90 from: https://chromedriver.storage.googleapis.com/114.0.5735.90/chromedriver_mac64.zip progress: 0/8693987 (0%) progress: 1684330/8693987 (19%) progress: 6124394/8693987 (70%) progress: 8693987/8693987 (100%) Unpacking ChromeDriver 114.0.5735.90 into /Users/ashwini/Desktop/VBCS/Automation/vbcs-client/visual-builder/breeze/vscode/extension/test-resources

So eventually the tests fails as my browser support is for 119 and driver downloaded is for 114. Hoping to see a solution!!

[djelinek]

download of chromedriver 114.x.x.x is correct. It is automatically picked from used VS Code version for testing. So for VS Code 1.84.2 it is on top of Chromium 114.x.x.x which we just get corresponding chromedriver to that chromium version.

anyway from logs it looks your tests are trying to run against your local chrome binary and not with vscode chrome app and downloaded chromedriver inside test-resources folder... before I will investigate more I have noticed you are running tests with NodeJS 18.16.1 and vscode-extension-tester 5.5.2

Could you please try to reproduce with latest vscode-extension-tester 6.0.0 ? and also with NodeJS 18.15.0 (last tested and supported NodeJS working with selenium-webdriver) ?

[AshwiniJRM]

Sure, I will check and keep you posted on the latest vscode-extension-tester with relevant nodeJS support. But is there a command to install vscode chrome app instead of using my local chrome binary and add to rest-resources folder in my case?

[djelinek]

if everything goes well it should work that way by default and not to use your local chrome binary but just resources downloaded by extest.. if the problem persists after you try with "correct" version of extester and Node I will need to reproduce with snippet you have provided

[AshwiniJRM]

I did update vscode-extension-tester to 6.0.0 with NodeJS to 18.15.0 and NPM to 9.5.0, and still see the same error as it is launching my local chrome browser (version 119) and not the defaulted browser within VSCode (version 114, specified by chromedriver version).

Request you to take a look at it and Kindly let me know if there is any alternate solution!!

[AshwiniJRM]

Is there anyother alternate solution that I can try?

[djelinek]

Hello @AshwiniJRM , I am sorry for late response, I was on PTO.

Could you please retry with a bit modified script for tests execution? unfortunately I cannot try locally without project and VSIX so please let me know all the outputs you can get.

"preui-test": "npm run transpile-tests-to-commonjs && rm -rf test-res && rm -rf test-resource && rm -rf ./tests/webdriver/_fixtures/tmp*",
"ui-test": "extest setup-tests --extensions_dir .test-extensions && extest install-vsix --vsix_file ./test-resources/**.vsix --extensions_dir .test-extensions && extest run-tests --mocha_config ./.mocharc.json --extensions_dir .test-extensions './out/tests/webdriver/tests/**/*.js'"

I am not also convinced about ./test-resources/**.vsix. I would also give a try with a full name of VSIX file.

[AshwiniJRM]

Sorry for the late response. I have run through the commands stated by you, and please find the output details -

1. $ npm run transpile-tests-to-commonjs && rm -rf test-res && rm -rf test-resource && rm -rf ./tests/webdriver/_fixtures/tmp*

   visual-builder-studio-vscode-extension@1.0.0-WillBeChangedByVSIXBuild preui-test npm run transpile-tests-to-commonjs && rm -rf test-res && rm -rf test-resource && rm -rf ./tests/webdriver/_fixtures/tmp* visual-builder-studio-vscode-extension@1.0.0-WillBeChangedByVSIXBuild transpile-tests-to-commonjs node_modules/.bin/babel ./out/tests -d ./out/tests --config-file ./test-babel.config.json Successfully compiled 27 files with Babel (1600ms).

---

2. $ extest setup-tests --extensions_dir .test-extensions && extest install-vsix --vsix_file ./test-resources/.vsix --extensions_dir .test-extensions && extest run-tests --mocha_config ./.mocharc.json --extensions_dir .test-extensions './out/tests/webdriver/tests//*.js'

   visual-builder-studio-vscode-extension@1.0.0-WillBeChangedByVSIXBuild preui-test npm run transpile-tests-to-commonjs && rm -rf test-res && rm -rf test-resource && rm -rf ./tests/webdriver/_fixtures/tmp visual-builder-studio-vscode-extension@1.0.0-WillBeChangedByVSIXBuild transpile-tests-to-commonjs node_modules/.bin/babel ./out/tests -d ./out/tests --config-file ./test-babel.config.json Successfully compiled 27 files with Babel (1478ms). visual-builder-studio-vscode-extension@1.0.0-WillBeChangedByVSIXBuild ui-test extest setup-tests --extensions_dir .test-extensions && extest install-vsix --vsix_file ./test-resources/.vsix --extensions_dir .test-extensions && extest run-tests --mocha_config ./.mocharc.json --extensions_dir .test-extensions './out/tests/webdriver/tests//.js' Downloading VSCode: 1.85.2 / stable VS Code exists in local cache, skipping download ChromeDriver 114.0.5735.90 exists in local cache, skipping download Error: Couldn't detect the repository where this extension is published. The link '../../vbcs-client/README.md#how-to-develop' will be broken in README.md. GitHub/GitLab repositories will be automatically detected. Otherwise, please provide the repository URL in package.json or use the --baseContentUrl and --baseImagesUrl options. at urlReplace (/Users/ashwini/Desktop/VBCS/Automation/vbcs-client/visual-builder/breeze/vscode/extension/node_modules/@vscode/vsce/out/package.js:511:31) at String.replace () at ReadmeProcessor.onFile (/Users/ashwini/Desktop/VBCS/Automation/vbcs-client/visual-builder/breeze/vscode/extension/node_modules/@vscode/vsce/out/package.js:522:33) at async Promise.all (index 686) at async pack (/Users/ashwini/Desktop/VBCS/Automation/vbcs-client/visual-builder/breeze/vscode/extension/node_modules/@vscode/vsce/out/package.js:1295:19) at async packageCommand (/Users/ashwini/Desktop/VBCS/Automation/vbcs-client/visual-builder/breeze/vscode/extension/node_modules/@vscode/vsce/out/package.js:1314:36)

---

3. $extent setup-tests gives me this error. But I can kick start the execution when I run "extest get-vscode && extest get-chromedriver && extest install-vsix --vsix_file ./.vsix && extest run-tests --mocha_config ./.mocharc.json out/tests/webdriver/tests//*.js". VSCode triggers the test and below is the output ->

   visual-builder-studio-vscode-extension@1.0.0-WillBeChangedByVSIXBuild preui-test npm run transpile-tests-to-commonjs && rm -rf test-res && rm -rf test-resource && rm -rf ./tests/webdriver/_fixtures/tmp visual-builder-studio-vscode-extension@1.0.0-WillBeChangedByVSIXBuild transpile-tests-to-commonjs node_modules/.bin/babel ./out/tests -d ./out/tests --config-file ./test-babel.config.json Successfully compiled 27 files with Babel (933ms). visual-builder-studio-vscode-extension@1.0.0-WillBeChangedByVSIXBuild ui-test extest get-vscode && extest get-chromedriver && extest install-vsix --vsix_file ./.vsix && extest run-tests --mocha_config ./.mocharc.json out/tests/webdriver/tests//.js Downloading VSCode: 1.85.2 / stable VS Code exists in local cache, skipping download ChromeDriver 114.0.5735.90 exists in local cache, skipping download Installing extensions... (node:98830) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead. (Use Electron --trace-deprecation ... to show where the warning was created) Extension 'visual-builder-studio-vscode-extension-2401.0.0+build-2023.12.12-15.20.vsix' was successfully installed. Loading mocha configuration from ./.mocharc.json Writing code settings to /Users/ashwini/Desktop/VBCS/Automation/vbcs-client/visual-builder/breeze/vscode/extension/test-resources/settings/User/settings.json Launching browser... Browser ready in 10078 ms Launching tests...

[AshwiniJRM]

Can I know the path of the local driver that VSCode holds? Based on the VSCode version, it downloads the binary of the driver. So I was thinking if we also have an executable file of the driver, I can think of passing that executable path and check if it gets picked while running.

[jkovalsky]

@djelinek, did you have time to take a look at this?

[djelinek]

Can I know the path of the local driver that VSCode holds? Based on the VSCode version, it downloads the binary of the driver. So I was thinking if we also have an executable file of the driver, I can think of passing that executable path and check if it gets picked while running.

the chromedriver binary is available in test-resources folder

Unpacking ChromeDriver 114.0.5735.90 into /Users/ashwini/Desktop/VBCS/Automation/vbcs-client/visual-builder/breeze/vscode/extension/test-resources

[djelinek]

@AshwiniJRM what mac arch are you running tests on? arm or intel?

[AshwiniJRM]

I am on mac intel.

the chromedriver binary is available in test-resources folder I was curious to understand why the vscode latest versions have corresponding chromium version which is still with 114 and nothing greater than that. So I am right now trying to check if I can get the latest chromium that can be used to download the corresponding chrome version matching with the binary.

[AshwiniJRM]

Even the latest VSCode (1.86.0) has chrome-118. Is there a possibility to change the chromium version tied to VSCode, so that I can use the latest chrome (121)? Does VSCode give us the liberty to change the chrome/firefox version other than what is tied to it?

[djelinek]

Even the latest VSCode (1.86.0) has chrome-118. Is there a possibility to change the chromium version tied to VSCode, so that I can use the latest chrome (121)? Does VSCode give us the liberty to change the chrome/firefox version other than what is tied to it?

I don't think so. VS Code is tied up always with Chromium version which you need to pickup appropriate chrome/chromedriver version to test with

[hellyzh]

@djelinek, get the same issue, looks like this code getting an empty file if chrome driver version > 114, chrome driver version is not set to the return value https://github.com/redhat-developer/vscode-extension-tester/blob/main/packages/extester/src/util/driverUtil.ts#L161

[djelinek]

unfortunately I am not able to reproduce. For VS Code 1.87 the chromedriver is v118, for VS COde 1.88 is the chromedriver v120 and for both options it seems that everything is working

[hellyzh]

@djelinek Thanks. Yes, it is working after upgrading to latest version.