[🐛 Bug]: selenium-manager disregards --driver-mirror-url when retrieving the geckodriver index document

BernMcCarty commented 2 months ago

What happened?

Unlike edge.rs and chrome.rs, firefox.rs contains a hardcoded URL for the driver index document (see the constant DRIVER_VERSIONS_URL). One could consider this alright if you consider only the fact that the index document contains no URLs that require transformation to point to the specified driver mirror, however, if in an environment that blocks access to githubusercontent.com, that alone could be the motivation to use a local mirror in the first place. That is our situation. We have set up a local mirror and it works for chrome and edge, but it does not work for firefox due to this bug.

How can we reproduce the issue?

PS C:\Users\auser\selenium-manager\windows> .\selenium-manager.exe --cache-path C:\temp\se --driver GeckoDriver --driver-mirror-url <your-gecko-mirror-url-here> --os windows --arch x64 --debug --trace --avoid-browser-download

Relevant log output

[2024-05-14T14:34:17.467Z TRACE] Reading metadata from C:\temp\se\se-metadata.json
[2024-05-14T14:34:17.472Z TRACE] Metadata file does not exist. Creating a new one
[2024-05-14T14:34:17.473Z DEBUG] Sending stats to Plausible: Props { browser: "firefox", browser_version: "", os: "windows", arch: "x64", lang: "", selenium_version: "4.20" }
[2024-05-14T14:34:17.474Z TRACE] Writing metadata to C:\temp\se\se-metadata.json
[2024-05-14T14:34:17.477Z TRACE] Checking geckodriver in PATH
[2024-05-14T14:34:17.534Z DEBUG] geckodriver not found in PATH
[2024-05-14T14:34:17.535Z DEBUG] firefox detected at C:\Program Files\Mozilla Firefox\firefox.exe
[2024-05-14T14:34:17.537Z TRACE] Path C:\Program Files\Mozilla Firefox\firefox.exe has been escaped to C:\\Program Files\\Mozilla Firefox\\firefox.exe
[2024-05-14T14:34:17.542Z TRACE] Using shell command to find out firefox version
[2024-05-14T14:34:17.543Z DEBUG] Running command: wmic datafile where name='C:\\Program Files\\Mozilla Firefox\\firefox.exe' get Version /value
[2024-05-14T14:34:17.811Z DEBUG] Output: "\r\r\n\r\r\nVersion=125.0.2.8875\r\r\n\r\r\n\r\r\n\r"
[2024-05-14T14:34:17.813Z TRACE] The version of firefox is 125.0.2.8875
[2024-05-14T14:34:17.814Z DEBUG] Detected browser: firefox 125.0.2.8875
[2024-05-14T14:34:17.819Z TRACE] Reading metadata from C:\temp\se\se-metadata.json
[2024-05-14T14:34:18.060Z WARN ] Problem reading geckodriver versions: Error parsing JSON from URL https://raw.githubusercontent.com/SeleniumHQ/selenium/trunk/common/geckodriver/geckodriver-support.json expected value at line 1 column 1. Using latest geckodriver version
[2024-05-14T14:34:18.343Z TRACE] Writing metadata to C:\temp\se\se-metadata.json
[2024-05-14T14:34:18.346Z ERROR] The geckodriver version cannot be discovered

Operating System

Windows 10

Selenium version

4.20.0

What are the browser(s) and version(s) where you see this issue?

All GeckoDriver versions

What are the browser driver(s) and version(s) where you see this issue?

All firefox versions that include selenium-manager

Are you using Selenium Grid?

No response

github-actions[bot] commented 2 months ago

@BernMcCarty, thank you for creating this issue. We will troubleshoot it as soon as we can.

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

github-actions[bot] commented 2 months ago

This issue is looking for contributors.

Please comment below or reach out to us through our IRC/Slack/Matrix channels if you are interested.

atmnk commented 1 month ago

I would like to contribute to this issue. I went thought the code and it looks like the hard coded url is mapping between geckodriver version and supported firefox version. Are we sure this information will be available in other mirrors? because when i went through other ( this ) mirror https://registry.npmmirror.com/-/binary/geckodriver it wasn't. @BernMcCarty are you sure your mirror has this information? Also if geckodriver mirrors are following any standard with this information we can make this committed file in selenium repo to follow that standard and then change the implementation to respect that standard and flag and fallback.

diemol commented 1 month ago

@bonigarcia do you know if this is implemented or not?

bonigarcia commented 1 month ago

The module for the Firefox features (firefox.rs) already supports mirroring URLs for drivers (https://github.com/mozilla/geckodriver/releases/) and browsers (https://ftp.mozilla.org/pub/firefox/releases/).

Nevertheless, the file for mapping browser and driver versions is located at a different URL (https://raw.githubusercontent.com/SeleniumHQ/selenium/trunk/common/geckodriver/geckodriver-support.json), and that URL does not currently support a mirror.

Implementing a mirror flag for that URL would require including a new flag (different from --driver-mirror-url).

BernMcCarty commented 1 month ago

FWIW I implemented a mirror as a specialized proxy. @bonigarcia I do not really see why this would necessarily require enlarging the CLI just for firefox. If firefox.rs would just retrieve the index document via the value of --driver-mirror-url (if specified), perhaps with /geckodriver-support.json catenated to it, then that would work for me.

SeleniumHQ / selenium