Using under Docker, using Arm images, on M1 Mac causes error

tonybyrne commented 3 years ago

Summary

Docker Desktop for Mac on M1, by default, where possible pulls Linux images to match the M1 architecture. When such an image uses the webdrivers gem, it downloads and attempts to use a Linux chromedriver built for x86 CPUs. This causes an error when the gem tries to launch the driver. Note that the repository of chromedrivers from which webdrivers pull the driver only has linux-x86 compiled drivers, so it's not clear if this can be solved.

Debug Info

Please provide the following information for bug reports:

Webdrivers version: Any latest version
Ruby version: 2.6.6, but I'd imagine all versions would be affected
Operating system / CI Environment: Docker Desktop 3.3.3 under Big Sur on an M1 Macbook Pro
Browser and version: Chrome (any latest version)

Expected Behavior

webdrivers should use a chromedriver built for Linux on an arm CPU.

Actual Behavior

An error is thrown and execution is aborted:

/lib64/ld-linux-x86-64.so.2: No such file or directory

ilynne commented 3 years ago

We are having this issue on M1 Macs, too

kapoorlakshya commented 3 years ago

This looks like a valid use case that we could consider supporting in the future, but I, unfortunately, don't have a M1 mac to investigate/troubleshoot or fix this. If the chromedriver downloads site starts providing arm binaries, then this could be easy to support. I know the electron project releases various arm binaries, but their versioning does not match the Chrome versioning, which is what webdrivers uses to determine which driver version to download.

If anyone has any ideas or is interested in submitting a PR for this, I'd be happy to collaborate and/or review PR(s).

agodwin commented 3 years ago

Not sure if this is related, but I have the new mac-mini with the M1 processor. I used RVM to install Ruby 2.6. When I ran 'webdrivers' to install chromedriver, I got the x86 version and the install failed with: Errno::EBADARCH: Bad CPU type in executable - /Users/tgodwin/.webdrivers/chromedriver

For some reason, when I used RVM to install 2.7, the 'webdrivers' gem installed the correct chromedriver.

I traced the problem to the value of RUBY_PLATFORM. For my Ruby 2.6 install the value is '-darwin20' and for Ruby 2.7 the value is 'arm64-darwin20' This caused 'Webdrivers::System.apple_m1_architecture?' to return false for 2.6 and true for 2.7 (see line 152 in '/webdrivers/blob/master/lib/webdrivers/system.rb).

I googled and read that RUBY_PLATFORM is set by 'rbconfig.rb.' I don't know much about rbconfig.rb. It seems that RVM failed to configure it correctly for 2.6. When I looked for 'rbconfig.rb' file on my system, I saw this: % find ~/.rvm -name rbconfig.rb /Users/tgodwin/.rvm/rubies/ruby-2.7.2/lib/ruby/2.7.0/arm64-darwin20/rbconfig.rb /Users/tgodwin/.rvm/rubies/ruby-2.6.6/lib/ruby/2.6.0/-darwin20/rbconfig.rb /Users/tgodwin/.rvm/src/ruby-2.7.2/rbconfig.rb /Users/tgodwin/.rvm/src/ruby-2.6.6/rbconfig.rb

I need to run Ruby version 2.6, so my solution was to hack 'Webdrivers::System.apple_m1_architecture?' in '/Users/tgodwin/.rvm/gems/ruby-2.6.6/gems/webdrivers-4.6.0/lib/webdrivers/system.rb' to return true. This way I can still use webdrivers to keep my chromedriver up to date:

% gem list webdrivers LOCAL GEMS webdrivers (4.6.0, 4.5.0)

% uname -prm 20.5.0 arm64 arm

Hope this helps.

p.s. I am not running Docker. This is all about native MacOS with the M1 processor.

kapoorlakshya commented 3 years ago

@agodwin Don't think your issue is related to the main one, but we can address it anyway. Just to be sure, could you also confirm if installing Ruby 2.6 with rbenv yields the same results? If it does, then could you please put in a PR with the fix you mentioned? I agree that updating System. apple_m1_architecture is all we'd need.

agodwin commented 2 years ago

@kapoorlakshya As much as I want to help, I am not going to take the time to retest with rbenv. Best of luck

nicalpi commented 2 years ago

If someone is passing by here with the same issue, I can confirm that the Ruby version, for me was in case.

I use rbenv, and the project I was working on used ruby 2.5.

Running the test, using selenium-webdrivers and webdrivers gem were resulting in a Errno::EBADARCH: Bad CPU type in executable error on an M1.

Updating to Ruby 2.7, makes the error go away.

titusfortner commented 2 years ago

Sounds like the solution is to upgrade to 2.7 If something else needs to be done we can reopen the issue. Thanks!

abineetds commented 2 years ago

Hello! I'm still running into this issue, I'm using ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [aarch64-linux]

LeEnno commented 1 year ago

Sounds like the solution is to upgrade to 2.7

This only relates to native macOS without docker.

The issue still persists for linux arm64 images which is the reason the issue was opened in the first place. Not sure if this makes this issue a duplicate of #242.

titusfortner commented 1 year ago

I'll reopen while we check it out.

I've been working on how to get all of this functionality directly into Selenium. Ideally we won't need a separate project to manage drivers in the near future.

Dan2552 commented 1 year ago

Does anybody have any individual winning combination (chrome version + electron version) using the electron releases? Maybe at least as a semi-workable solution webdrivers could contain a hardcoded hash of known working version couples?

LeEnno commented 1 year ago

chromedriver-v21.2.0-linux-arm64.zip from electron v21.2.0 release page installs ChromeDriver 106.0.5249.119.
chromium-browser --version says Chromium 107.0.5304.62 Ubuntu 20.04

manishElitmus commented 1 year ago

@titusfortner Any updates on this?

titusfortner commented 1 year ago

I'm much more interested in getting things working with Selenium going forward. Can you update to Selenium 4.11, remove this gem and we can figure out what it needs to do that it isn't?

manishElitmus commented 1 year ago

I'm much more interested in getting things working with Selenium going forward. Can you update to Selenium 4.11, remove this gem and we can figure out what it needs to do that it isn't?

Hi @titusfortner

I tried it with selenium-webdriver 4.11.0 using docker on arm64/linux machine

Chromdriver is installed successfully at location /root/.cache/selenium/chromedriver/linux64/116.0.5845.96/chromedriver

But it's not able to connnect to the driver

E

Error:
BlogsTest#test_should_update_Blog:
Selenium::WebDriver::Error::WebDriverError: unable to connect to /root/.cache/selenium/chromedriver/linux64/116.0.5845.96/chromedriver 127.0.0.1:9515
    test/system/blogs_test.rb:26:in `block in <class:BlogsTest>'

Error:
BlogsTest#test_should_update_Blog:
Selenium::WebDriver::Error::WebDriverError: unable to connect to /root/.cache/selenium/chromedriver/linux64/116.0.5845.96/chromedriver 127.0.0.1:9515
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/service_manager.rb:132:in `connect_until_stable'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/service_manager.rb:58:in `block in start'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/socket_lock.rb:42:in `locked'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/service_manager.rb:55:in `start'
    <internal:kernel>:90:in `tap'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/service.rb:90:in `launch'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/driver.rb:334:in `service_url'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/local_driver.rb:28:in `initialize_local_driver'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/chrome/driver.rb:34:in `initialize'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/driver.rb:47:in `new'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/driver.rb:47:in `for'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver.rb:88:in `for'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara/selenium/driver.rb:83:in `browser'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/driver.rb:64:in `block in register_selenium'
    <internal:kernel>:90:in `tap'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/driver.rb:63:in `register_selenium'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/driver.rb:49:in `block in register'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara/session.rb:105:in `driver'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara/session.rb:91:in `initialize'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara.rb:421:in `new'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara.rb:421:in `block in session_pool'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara.rb:317:in `current_session'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara/dsl.rb:46:in `page'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/test_helpers/screenshot_helper.rb:99:in `save_image'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/test_helpers/screenshot_helper.rb:36:in `take_screenshot'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/test_helpers/screenshot_helper.rb:45:in `take_failed_screenshot'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/test_helpers/setup_and_teardown.rb:8:in `before_teardown'

What I am doing wrong here?

manishElitmus commented 1 year ago

I'm much more interested in getting things working with Selenium going forward. Can you update to Selenium 4.11, remove this gem and we can figure out what it needs to do that it isn't?

Hi @titusfortner

I tried it with selenium-webdriver 4.11.0 using docker on arm64/linux machine

Chromdriver is installed successfully at location /root/.cache/selenium/chromedriver/linux64/116.0.5845.96/chromedriver

But it's not able to connnect to the driver

E

Error:
BlogsTest#test_should_update_Blog:
Selenium::WebDriver::Error::WebDriverError: unable to connect to /root/.cache/selenium/chromedriver/linux64/116.0.5845.96/chromedriver 127.0.0.1:9515
    test/system/blogs_test.rb:26:in `block in <class:BlogsTest>'

Error:
BlogsTest#test_should_update_Blog:
Selenium::WebDriver::Error::WebDriverError: unable to connect to /root/.cache/selenium/chromedriver/linux64/116.0.5845.96/chromedriver 127.0.0.1:9515
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/service_manager.rb:132:in `connect_until_stable'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/service_manager.rb:58:in `block in start'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/socket_lock.rb:42:in `locked'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/service_manager.rb:55:in `start'
    <internal:kernel>:90:in `tap'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/service.rb:90:in `launch'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/driver.rb:334:in `service_url'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/local_driver.rb:28:in `initialize_local_driver'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/chrome/driver.rb:34:in `initialize'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/driver.rb:47:in `new'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver/common/driver.rb:47:in `for'
    /usr/local/bundle/gems/selenium-webdriver-4.11.0/lib/selenium/webdriver.rb:88:in `for'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara/selenium/driver.rb:83:in `browser'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/driver.rb:64:in `block in register_selenium'
    <internal:kernel>:90:in `tap'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/driver.rb:63:in `register_selenium'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/driver.rb:49:in `block in register'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara/session.rb:105:in `driver'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara/session.rb:91:in `initialize'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara.rb:421:in `new'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara.rb:421:in `block in session_pool'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara.rb:317:in `current_session'
    /usr/local/bundle/gems/capybara-3.39.2/lib/capybara/dsl.rb:46:in `page'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/test_helpers/screenshot_helper.rb:99:in `save_image'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/test_helpers/screenshot_helper.rb:36:in `take_screenshot'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/test_helpers/screenshot_helper.rb:45:in `take_failed_screenshot'
    /usr/local/bundle/gems/actionpack-7.0.7/lib/action_dispatch/system_testing/test_helpers/setup_and_teardown.rb:8:in `before_teardown'

What I am doing wrong here?

When I navigated to the the chromedriver path and executed the chromedriver file there. It's give the same error as mentioned above in the issue description.

It seems it still dosen't support on linux/arm64 machine.

titusfortner commented 1 year ago

Which driver needs to be downloaded to work on your OS? I kind of think this needs to be an issue raised with Google if they don't offer one. Does Edge have one?

manishElitmus commented 1 year ago

I am trying to use google chrome browser with the Chromedriver.

I'll check if it's working with Edge or not. And let you know

titusfortner commented 1 year ago

Right, does google provide a driver that works for Linux/Arm? Do you have something working manually that you want provided automatically, or are you just looking for something that works?

manishElitmus commented 1 year ago

Right, does google provide a driver that works for Linux/Arm? Do you have something working manually that you want provided automatically, or are you just looking for something that works?

I attempted the installation of the chromedriver on Debian using the following link: https://packages.debian.org/buster/chromium-driver. The installation was successful, and the chromedriver is now located at the path /usr/bin/chromedriver. It is functioning properly.

However, I encountered an issue while configuring it with selenium-webdriver version 4.11.0. The problem stems from a version discrepancy between the Chrome browser and the chromedriver. This difference is noticeable in the selenium-manager.json file.

On the other hand, I achieved success with selenium-webdriver version 4.6.1. By specifying the driver path as Selenium::WebDriver::Chrome::Service.driver_path = '/usr/bin/chromedriver', I was able to solve issues and attain proper functionality

titusfortner commented 1 year ago

Yeah, there's some concern about how to manage drivers that aren't provided by the browser vendor directly I think the requirement needs to be that Google support it — https://github.com/GoogleChromeLabs/chrome-for-testing#:~:text=the%20current%20list%20of%20supported%20platforms%20is

The alternative manually updating the known-good driver version as necessary and putting them in a location on PATH as you are doing. You can set the browser not to automatically update versions, or use Chrome for Testing binaries which won't auto update.

manishElitmus commented 11 months ago

I had written a article on how to solve this problem, anyone interested please give a read.

https://www.elitmus.com/blog/technology/demystifying-rails-7-system-tests-configuring-ci-pipeline/

cc: @titusfortner

titusfortner / webdrivers