Using Capybara with headless chrome

[twalpole]

There are currently 2 issues with using Capybara with headless chrome -

(Session info: headless chrome=60.0.3080.5) (Driver info: chromedriver=2.29.461585

1. Headless chrome appears not to support JS system modals ( alert, confirm, prompt) There is a workaround for this currently in testing

2. Attempting to close a window raises a timeout error "failed to close window in 20 seconds" and doesn't close the window I can't think of any way to work around this issue, so window management won't really work until this is fixed in either chromedriver or chrome.

[twalpole]

The workaround for JS system modals is PR #1859 which was merged into master - 12c10059709346bfeadb2cea18f510174d842303

[jesperronn]

Could you please elaborate on how I set up and run with headless chrome?

I am curious to how you tested, and I'm eager to test myself

[twalpole]

@jesperronn The setup Capybara uses for it's headless chrome tests is https://github.com/teamcapybara/capybara/blob/master/spec/selenium_spec_chrome.rb#L6

You should just need to pass args: ['headless'] as an option in your driver registration

[gregsadetsky]

Hi,

I'm not involved in / don't use capybara (although I've heard great things) :-) but I did run into the same issues with headless Chrome while using Selenium using Python (alerts not being supported & driver.close() causing an issue)

Notes on how I resolved both:

• I did implement the same approach as you did for alerts (#1859 - overriding window.alert to set a globally accessible variable and then checking if that variable had been set).

One difference is that we used a random variable name/value (in our single page app, setting a global "alert() has been called" variable could be a false positive as the 'true' value could be read a 2nd, 3rd, etc. time)

The second difference is that we ended up setting a cookie in the window.alert handler instead of setting a global variable -- the reason being, if you have an alert() call closely followed by a location.href change, the variable used to track if the alert happened or not will be lost; this won't happen with a cookie (we did use a random value for the cookie, for reasons explained above).

• As for driver.close() causing "failed to close window in 20 seconds" errors, the solution was simpler: instead of using the 'stable' Chrome channel, we switched to the 'beta' channel. The current Linux version of Chrome Beta is 59.0.3071.83.

Cheers!

[twalpole]

@gregsadetsky Thanks for the info. The alert/prompt/confirm workaround was meant to be an easy solution until Chrome/chromedriver fixed the issue, however it looks like I will need to make it more robust since Chrome 59 has released with the issue still there.

[twalpole]

@gregsadetsky Hmmm, I still see the window errors on MacOS with Chrome 59.0.3071.86 and chromedriver 2.29.461585 so it may be fixed in linux, but it's not fully fixed.

[twalpole]

@gregsadetsky and on linux (travis) we're seeing a different error now -

Selenium::WebDriver::Error::UnknownError: unknown error: cannot get automation extension from unknown error: page could not be found: chrome-extension://aapnijgdinlhnhlmodcfapnahmbfebeb/_generated_background_page.html (Session info: headless chrome=59.0.3071.86) (Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 4.4.0-51-generic x86_64)

Any experience with that error?

[gregsadetsky]

Unfortunately, no. It seems like that version (59.0.3071.86) will be rolling out to the stable channel (we're successfully using 59.0.3071.83 on the beta channel). Ugh. :-)

[twalpole]

@gregsadetsky "Ugh", yeah. Wrt your comments about random variable names, after taking a quick look at my implementation again, I don't think it applies to Capybara. I create a new "modal handler" instance every time the user tells us there is going to be an alert/prompt/confirm and then remove it from the queue when the status is checked so it's not really possible for the same status to be used multiple times. The issue with a page change is valid, but swapping to cookies could also have an issue if cookies are cleared during the page change (or the new url is a different sub/domain) so I think I'll stay with the small risk of a failure if a page change occurs, for now, and hope chrome/chromedriver fix the issue soon.

[openscript]

Here is the new chromedriver 2.30. It doesn't have release notes yet. https://chromedriver.storage.googleapis.com/index.html?path=2.30/

[jeremy]

Note that window sizing and positioning do not work with headless as of chromedriver 2.30 + Chrome 59.

From this chromedriver issue:

As part of the work we're doing to make Headless Chrome work with ChromeDriver, we're replacing the ChromeDriver automation extension with DevTools commands to control the window size. This should make the extension unnecessary, so we won't have to worry about whitelists.

If you're doing something like:

driver.browser.manage.window.size = Selenium::WebDriver::Dimension.new(1400, 1400)

Then you'll get an error like:

Selenium::WebDriver::Error::UnknownError: unknown error: cannot get automation extension
from unknown error: page could not be found: chrome-extension://aapnijgdinlhnhlmodcfapnahmbfebeb/_generated_background_page.html
  (Session info: headless chrome=59.0.3071.86)
  (Driver info: chromedriver=2.30.477690 (c53f4ad87510ee97b5c3425a14c0e79780cdf262),platform=Mac OS X 10.13.0 x86_64)

Comment out the window size/position setting to work around it.

[twalpole]

chromedriver 2.30 fixed the issues around window closing, but all content in extra windows opened is reported as not displayed by selenium, so multiple windows are still not really usable with headless.

[nertzy]

https://github.com/teamcapybara/capybara/blob/3879c8cdd03759975f27c29e3b4588438bdb95b6/lib/capybara/selenium/driver.rb#L311

This line makes a lot of assumptions about the hash structure of the Capabilities object.

I'm on a project where we have this driver defined:

Capybara.register_driver :chrome_headless do |app|
  capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
    chromeOptions: {
      args: %w[ no-sandbox headless disable-gpu ]
    }
  )

  Capybara::Selenium::Driver.new(app, browser: :chrome, desired_capabilities: capabilities)
end

And page.driver.headless_chrome? returns false because we have :chromeOptions instead of :chrome_options and we have :args instead of 'args'.

But Chrome does launch headlessly so it's clear that the browser launcher is being more lenient in its hash parsing.

[twalpole]

@nertzy Yes it does, and if you'd like to propose a clean way of detecting it a PR would be appreciated. The fact that we even have to care whether it's headless or not is a hack at the moment, and hopefully modals and window interactions will actually be supported by Chrome in the near future, so we don't have to care.

[iggant]

1) client authentication client addle to access dashboard with authentication Failure/Error: fill_in 'user[email]', with: client.user.email Selenium::WebDriver::Error::UnknownError: unknown error: an X display is required for keycode conversions, consider using Xvfb (Session info: headless chrome=59.0.3071.86) (Driver info: chromedriver=2.30.477691 (6ee44a7247c639c0703f291d320bdf05c1531b57),platform=Linux 4.9.24-coreos x86_64)

     # /usr/local/bundle/gems/selenium-webdriver-3.4.3/lib/selenium/webdriver/remote/response.rb:69:in `assert_ok'
     # /usr/local/bundle/gems/selenium-webdriver-3.4.3/lib/selenium/webdriver/remote/response.rb:32:in `initialize'
     # /usr/local/bundle/gems/selenium-webdriver-3.4.3/lib/selenium/webdriver/remote/http/common.rb:83:in `new'
     # /usr/local/bundle/gems/selenium-webdriver-3.4.3/lib/selenium/webdriver/remote/http/common.rb:83:in `create_response'
     # /usr/local/bundle/gems/selenium-webdriver-3.4.3/lib/selenium/webdriver/remote/http/default.rb:107:in `request'
     # /usr/local/bundle/gems/selenium-webdriver-3.4.3/lib/selenium/webdriver/remote/http/common.rb:61:in `call'
     # /usr/local/bundle/gems/selenium-webdriver-3.4.3/lib/selenium/webdriver/remote/bridge.rb:170:in `execute'
     # /usr/local/bundle/gems/selenium-webdriver-3.4.3/lib/selenium/webdriver/remote/oss/bridge.rb:579:in `execute'
     # /usr/local/bundle/gems/selenium-webdriver-3.4.3/lib/selenium/webdriver/remote/oss/bridge.rb:372:in `send_keys_to_element'
     # /usr/local/bundle/gems/selenium-webdriver-3.4.3/lib/selenium/webdriver/common/element.rb:156:in `send_keys'
     # /usr/local/bundle/gems/capybara-2.14.3/lib/capybara/selenium/node.rb:74:in `set'
     # /usr/local/bundle/gems/capybara-2.14.3/lib/capybara/node/element.rb:108:in `block in set'
     # /usr/local/bundle/gems/capybara-2.14.3/lib/capybara/node/base.rb:85:in `synchronize'
     # /usr/local/bundle/gems/capybara-2.14.3/lib/capybara/node/element.rb:106:in `set'
     # /usr/local/bundle/gems/capybara-2.14.3/lib/capybara/node/actions.rb:92:in `fill_in'
     # /usr/local/bundle/gems/capybara-2.14.3/lib/capybara/session.rb:784:in `block (2 levels) in <class:Session>'
     # /usr/local/bundle/gems/capybara-2.14.3/lib/capybara/dsl.rb:50:in `block (2 levels) in <module:DSL>'
     # ./spec/features/client_authentication_spec.rb:19:in `block (2 levels) in <top (required)>'

[twalpole]

@iggant That would be a Chrome issue, not a Capybara issue.

[deivid-rodriguez]

This one, maybe: https://bugs.chromium.org/p/chromedriver/issues/detail?id=1772

[twalpole]

@deivid-rodriguez Exactly that one :)

[frewsxcv]

This one, maybe: https://bugs.chromium.org/p/chromedriver/issues/detail?id=1772

Looks like this has been fixed and is just waiting for a new chromedriver release.

[lucascaton]

I'm able to run it and wrote a blog post about it:

How to run your feature specs using Capybara and Headless Chrome

[twalpole]

@lucascaton You've fixed what? Without the next release of chromedriver (2.31) it's not possible to run without an X server installed on linux, anything to do with multiple windows or window resizing is still pretty broken until a future release of chrome and/or chromedriver, and we're still hacking around the lack of JS modal support.

[lucascaton]

@twalpole I've been using it with chromedriver 2.30 and works perfectly, even on Circle CI, running the same version 😄

[twalpole]

@lucascaton Yes, because Circle CI installs an X server, your tests aren't resizing windows or opening multiple windows, and Capybara is hacking around the JS modals. So it's working perfectly for you because you're not using any of the currently broken parts. That's not fixing things, that's just avoiding the cracks :) Capybara has been running its own tests with headless chrome on travis for a few weeks now, and as long as we skip all the broken tests then it's perfect.

[himankarbn]

We are using Chromedriver 2.30 and the only issue we are facing is the resizing. Since our test suite does lot of resizing ( desktop, mobile, tablet sizes ) in a single feature spec, we are currently blocked in using headless feature. @jeremy @twalpole can you guys suggest how to use DevTools commands to control the window size during a test?

[twalpole]

@himankarbn Since there is no connection to send random DevTools commands over I believe this isn't possible to do at the moment. Basically, it's a waiting game until chromedriver/chrome implement/fix support.

[egbert]

If you need to resize just once you can set a flag for the window size instead of resizing the window:

Capybara.register_driver :chrome_headless do |app|
  capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
    chromeOptions: {
      args: %w[ no-sandbox headless disable-gpu window-size=1280,1024]
    }
  )

  Capybara::Selenium::Driver.new(app, browser: :chrome, desired_capabilities: capabilities)
end

[rilian]

we have sacrificed alerts functionality by disabling them with the following code, that is injected during tests:

window.alert = function() { return true; }
window.confirm = function() { return true; }

for everything else headless chrome works well.

[twalpole]

@rilian Alert functionality should be working with the latest Capybara and headless chrome - if it's not please let us know exactly what you're doing that doesn't work.

[himankarbn]

Thanks @egbert! We use shared examples and each of our feature spec executes on desktop, mobile, tablet etc views. The above solution on setting the window size is not much helpful for us.

[profsmallpine]

I tried moving from phantomjs to headless chrome today. There were a couple minor issues, but I am experiencing an issue where calling find('input#example').set('test') will sometimes work correctly. It also ends up setting the input as tes or te. Is anyone else experiencing this issue?

[twalpole]

@profsmallpine That kind of behavior is usually caused by JS behavior initializing and taking the focus while the keystrokes are being sent, or JS attached to the specific input not handling the speed keystrokes are sent at. You would need to wait for whatever JS initialization is happening to complete before sending the text, or fix/improve the JS attached to the input

[profsmallpine]

@twalpole That makes sense, thanks for the feedback! I will add that after getting about 50% flaky tests, I have not seen any failed tests for this reason after updating capybara from 2.14.2 -> 2.14.4.

[twalpole]

@profsmallpine hmm -- nothing different between those 2 releases that should affect this, did you update anything else at the same time?

[profsmallpine]

@twalpole The only change that I made was the capybara update in the commit where things seemed to get fixed, but it appears that I spoke too soon, still seeing the same flakiness, but only with chrome headless, phantomjs didn't experience this same issue.

[twalpole]

@profsmallpine The timing differences between phantomjs and chrome headless are going to be huge, so it's very possible that you have some JS behaviors attaching later under chrome and stealing focus. You can test that by just sleeping for a couple seconds before filling in the field in a test that is flaky. If that makes the flakiness stop then you'll need to figure out what visible change occurs on the page that you can check for before starting to fill it in, or fix the JS so it doesn't change the focus. Note, if that is the cause then it could/would affect a user that starts filling in the page immediately too, which would be a pretty bad UI.

[h-parekh]

@profsmallpine I'm using a 'sleep injector' that injects sleep into some of the Capybara actions. This way I don't need to have sleep statements in the scenario.

[twalpole]

@h-parekh Seleniums matchers and finders already have built-in waiting/retrying behavior so your overrides of wrap_matches?, has_selector?, and has_no_selector? are no different than just increasing the value of Capybara.default_max_wait_time or passing a large :wait option to the individual calls, so they're completely unneeded.

Your fill_in does make a change by waiting an extra 2 seconds after every fill_in but (if really needed) would be much better implemented as a module that gets prepended so you're not completely overwriting the existing method, something like (untested code but should be close)

module CapybaraSleepAdder
  def fill_in(*args)
     super.tap { sleep(2) }
  end
end
Capybara::Node::Actions.prepend(CapybaraSleepAdder)

[twalpole]

Chromedriver 2.31 has been released -- resizing/maximizing of windows, and visibility of content in windows other than the initial one appear to still be broken. Therefore selenium with chrome is still unusable in headless mode if you need to be resizing windows, or need to have multiple windows open at once.

[twalpole]

@Maxxillo Capybaras within_frame tests run fine in Chrome headless , so unless Google Captcha is detecting headless mode and doing something different it should work. Without access to a reproducible example it's not really possible to provide anymore info.

[maschwenk]

I had a React button with a onMouseEnter handler that was triggering just fine using hover on Polteregeist and now is not being triggered when using Chrome Headless. Anyone else had this issue?

Not sure at what level in the stack this would be happening, feel free to delete comment if this is out of scope of this Issue @twalpole

[twalpole]

@maschwenk Does it work with Chrome non-headless ? There are hover tests in the Capybara test suite that pass on both normal Chrome and Chrome headless, so hover definitely isn't fully broken with Chrome headless (doesn't mean it's not partly broken) - also how are you determining that it isn't triggering, timing between poltergeist and chrome headless will be VERY different.

Clarification: The capybara test suite only tests hover behavior with CSS -- it does not test onMouseEnter, it's definitely possible chromedriver isn't triggering an onMouseEnter event.

[maschwenk]

Basically by just using this to debug when the events are fired: page.driver.browser.manage.logs.get("browser")

I will try with non-headless Chrome. When you say timing will be very different, I assume you mean Chrome headless will be a lot slower?

[twalpole]

@maschwenk Slower and some things where Poltergeist waits for actions to complete that Chrome won't -- also see my clarification on previous answer about Capybaras tests for hover. Also, I assume logging to console as warn or higher (or have configured Chromes selenium logs to show lower severity than warn). In the past it only returned warn or higher IIRC, of course that might have changed recently (or I could be completely mistaken)

[maschwenk]

<button 
  onMouseEnter={() => console.error('Howdy doodly')} 
  onClick={() => console.error('Howdy doodly im a click!')}>
   Preview
</button>

find_button('Preview').hover

sleep 10
puts page.driver.browser.manage.logs.get('browser')

find_button('Preview').click

sleep 10
puts page.driver.browser.manage.logs.get('browser')

Hover does not does not print anything to console; click does.

UPDATE: Just saw your clarification. That makes sense. Because Selenium does not provide trigger I have no idea how I could possibly test this then?

[twalpole]

@maschwenk Ok -- sounds like it definitely isn't firing - does it work non-headless -- sounds like a deficiency in either chromedriver or Selenium.

[maschwenk]

Same issue with Chrome Head-more (?) Chrome Non-Headless (?)

[twalpole]

Ok -- sounds like a deficiency with either selenium or chromedriver, the webdriver spec only specifically mentions mouseover, and only in relation to option elements, so not sure whether they're supposed to be outputting a mouseenter (I would assume they should but the spec doesn't say) - You may be able to trigger the behavior through execute_script but since it happens with both Headless and Headed - it's not really this issue.

[fusionx1]

Hi Guys I just would like to know to know if it's possible to use the chrome browser session while I'm running my headless chrome. I just wanna skip login process and rely on the browser session. Thanks in advance,

[mooikos]

@twalpole As an example this code can work on non-headless driver but not working on headless driver :

page.visit('https://www.google.com/recaptcha/api2/demo')
page.within_frame(page.find('#recaptcha-demo iframe')) do
  page.find('.recaptcha-checkbox-checkmark').click
end

I tried to find documentation regarding this output (that comes after successfully clicking on the non-headless version) :

exit
=> Obsolete #<Capybara::Node::Element>#<Capybara::Node::Element:0x3fd7b39b06c0>
(pry) output error: #<NoMethodError: undefined method `[]=' for nil:NilClass>

There is no mention regarding "Obsolete" :(

[twalpole]

@fusionx1 Please don't hijaack this thread with howto questions. These issues are for bugs in Capybara and sometimes its related drivers. Howto questions should be asked on the mailing list, gitter, or stackoverflow