thoughtbot / capybara-webkit

A Capybara driver for headless WebKit to test JavaScript web apps
https://thoughtbot.com/open-source
MIT License
1.97k stars 428 forks source link

Capybara webkit crash on preflight CORS request failure #1055

Open AlanFoster opened 6 years ago

AlanFoster commented 6 years ago

When a preflight CORS request fails, capybara webkit crashes:

Capybara::Webkit::CrashError:
    The webkit_server process crashed!

     Connection reset by peer

    This is a bug in capybara-webkit. For help with this crash, please visit:

    https://github.com/thoughtbot/capybara-webkit/wiki/Reporting-Crashes

If it helps, this is the preflight options check that appears to kill webkit:

curl -v -X OPTIONS  --insecure https://api.github.com
* Rebuilt URL to: https://api.github.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connection failed
* connect to 127.0.0.1 port 49153 failed: Connection refused
* Failed to connect to localhost port 49153: Connection refused
* Closing connection 0
curl: (7) Failed to connect to localhost port 49153: Connection refused

Example

I have made the following project as an example:

git clone -b webkit-crashes-with-preflight-failure git@github.com:AlanFoster/rspec-capybara-feature-testing.git
cd rspec-capybara-feature-testing/website
bundle

Running the following command will cause webkit to crash 100% of the time:

$ FORCE_WEBKIT_CRASH=true DEBUG=true rspec ./spec/features/javascript_spec.rb
> ...
     Failure/Error: expect(page).to have_content('Successfully retrieved server response')

     Capybara::Webkit::CrashError:
       The webkit_server process crashed!

         Broken pipe

       This is a bug in capybara-webkit. For help with this crash, please visit:

       https://github.com/thoughtbot/capybara-webkit/wiki/Reporting-Crashes

Whilst correctly stubbing the preflight CORS request will not crash webkit:

DEBUG=true rspec ./spec/features/javascript_spec.rb
...

Finished in 2.15 seconds (files took 1.91 seconds to load)
1 example, 0 failures

Context: I am using puffing billy to stub requests made during feature tests.

AlanFoster commented 6 years ago

cc @twalpole - I've spent some time putting together an example of a consistent capybara webkit crash that you might like to see :+1:

twalpole commented 6 years ago

@AlanFoster That project appears to use headless chrome???? Is there a branch that uses capybara-webkit and did you configure it in debug mode?

AlanFoster commented 6 years ago

@twalpole Apologies, I should have clarified that the branch is webkit-crashes-with-preflight-failure :)

It's possible to replicate the crash with the following steps:

git clone -b webkit-crashes-with-preflight-failure git@github.com:AlanFoster/rspec-capybara-feature-testing.git
cd rspec-capybara-feature-testing/website
bundle

FORCE_WEBKIT_CRASH=true DEBUG=true rspec ./spec/features/javascript_spec.rb

The driver is capybara webkit, just with a configured proxy (Puffing Billy)

Let me know if i've missed anything :+1:

AlanFoster commented 6 years ago

To trigger the crash on that branch:

FORCE_WEBKIT_CRASH=true DEBUG=true rspec ./spec/features/javascript_spec.rb
twalpole commented 6 years ago

This seems to be a crash inside Qt -- not sure there's much capybara-webkit can do about it

AlanFoster commented 6 years ago

@twalpole Interesting! Thanks for looking into this :)

I was hoping to post more examples of errors that i've run into with Capyara Webkit, but I wonder if most of them are Qt's fault in the end - that would be interesting 🤔

Shall I close this issue, or raise a QT bug? :)

twalpole commented 6 years ago

There's really no point in raising a qt bug - qtwebkit isn't developed by the qt project anymore. There is an effort underway to revive it - https://github.com/annulen/webkit - and if you can get capybara-webkit to build with a modern Qt and the annullen version of qtwebkit and see if the issue still exists it would be a great move forward. I keep meaning to try and get that working, since it should allow capybara-webkit to support modern CSS/JS, but just haven't had time.

AlanFoster commented 6 years ago

@twalpole Thanks for the context!

Unfortunately I have no Qt experience and wouldn't even know where to begin! Having capybara-webkit support modern CSS/JS would be cool though! For what it's worth, running on an older JavaScript implementation has sometimes been useful catching JavaScript which wasn't transpiled correctly, or where we made use of newer functions that wouldn't work on Safari/IE for instance :)