search

thoughtbot / capybara-webkit

A Capybara driver for headless WebKit to test JavaScript web apps
https://thoughtbot.com/open-source
MIT License
1.97k stars 427 forks source link

Proxy Authentication using NTLM/Kerberos #882

Closed ghost closed 8 years ago

ghost commented 8 years ago

Hi @jferris,

I'm using capybara (2.6.2) and capybara-webkit (1.8.0) to build an rspec-extension called proxy_rb to do TDD of our HTTP proxy environments. Setting the proxy works fine. Authentication against proxies using the BASIC-scheme works fine as well.

BUT: In one of our testing environments we experiment with BASIC-, NTLM- and KERBEROS-authentication.

  1. Client sends request without any credentials

  2. Proxy answers with 407 Authentication required

    This response contains 3 realms which is allowed and therefor ok. The client can choose the realm it want to authenticate against:

    1. Basic
    2. NTLM
    3. Kerberos

    capybara-webkit chooses NTLM or Kerberos out of those. Unfortunately this fails as I run the test on a machine which is not part of the test windows domain.

I tried both Qt 4 and Qt 5.5.1. There's no change in behaviour.

WISHES:

  1. It would be great, if I could configure capybara-webkit to always use the BASIC-scheme

  2. It would be great, if NTLM-/Kerberos would work out of the box.

    Though I'm not sure, what is required to make this work with ruby + capybara + capybara-webkit on a Linux OS. Any hints about how to make it work are welcome

Cheers, Dennis

jferris commented 8 years ago

You're talking about a proxy you set up using config.set_proxy, right? The set_proxy method ends up in the SetProxy command, which creates an instance of QNetworkProxy with the provided host, port, username, and password. I don't see anything on QNetworkProxy that lets us control which realms or authentication methods it should use.

We're currently providing the username and password to the constructor in QNetworkProxy, but it looks like we might be able to instead listen for the proxyAuthenticationRequired signal, which lets us directly use a QAuthenticator, but I don't see a place to control the authentication from there, either.

The classes I linked to are the places I'd think to start poking around if you want to make this more configurable. I don't have time to work on this myself, but if you want to submit a pull request or have further questions, I'll check in again.

(I'm closing this issue, because it doesn't seem like a bug in capybara-webkit, but don't let that stop you from commenting again.)

ghost commented 8 years ago

You're talking about a proxy you set up using config.set_proxy, right?

Correct.

The classes I linked to are the places I'd think to start poking around if you want to make this more configurable. I don't have time to work on this myself, but if you want to submit a pull request or have further questions, I'll check in again.

Unfortunately I'm not a Qt-developer. Just for your interest. I filed an issue at the Qt project bug tracker. You can find it here https://bugreports.qt.io/browse/QTBUG-51639.

We're currently providing the username and password to the constructor in QNetworkProxy, but it looks like we might be able to instead listen for the proxyAuthenticationRequired signal, which lets us directly use a QAuthenticator, but I don't see a place to control the authentication from there, either.

I see. But I've no idea how to solve it properly either. I hope to get some feedback from the Qt project. This situation is quite annoying: Now I only put together a library whichs solves only 1/2 of the problem (HTTP Basic works fine, but HTTP NTLM doesn't).

What makes it worse: Both capybara-webkit and phantomjs (poltergeist) seem to use Qt in the background.

jferris commented 8 years ago

What makes it worse: Both capybara-webkit and phantomjs (poltergeist) seem to use Qt in the background.

Yeah, there aren't many options for a distributable browser engine. The most promising thing on the horizon looks like the Chromium Embedded Framework.

ghost commented 8 years ago

Yeah, there aren't many options for a distributable browser engine. The most promising thing on the horizon looks like the Chromium Embedded Framework.

But would it be really worth to switch the engine? Are there any more "quirks" making the use of webkit in the background annoying for you as a developer? I'm just curious.

jferris commented 8 years ago

But would it be really worth to switch the engine?

We've been investigating other options because QtWebKit is being deprecated. The Qt replacement, QtWebEngine, doesn't support the kind of work we're doing in capybara-webkit.

ghost commented 8 years ago

Ah. Ok. Thanks for the feedback.

ghost commented 8 years ago

Do you think doing NTLM/Kerberos with Chromium Embedded Framework will work out of the box and is more configurable? That would be great at least if the user can choose the REALM or authentication mechanism the authenticator will use.

jferris commented 8 years ago

Do you think doing NTLM/Kerberos with Chromium Embedded Framework will work out of the box and is more configurable?

I'm not sure, but my guess is not out of the box, no. Most of the network and proxy behavior we're currently using comes from Qt and not from WebKit. This has been great because it provides a lot of functionality without too much extra effort, but we frequently hit walls when it doesn't have explicit support for what we want. I'm not sure if CEF does anything for proxies out of the box, but we'll find out when we get further along. My guess is that CEF will involve more legwork but allow us to do more if somebody is willing to put in the time.

ghost commented 8 years ago

Good to know. Recently I got my gem to work as expected - see this feature-file (rspec) or this feature-file (cucumber) if you're interested in an example.

With the help of capybara-webkit I'm now able to run 2500 tests (requests) in a fast way against all our proxy servers. Only bigger responses (150MiB) make it core dump. :-) Thanks a lot for your work in capybara-webkit.

I hope you will keep "proxy"-support in the successor of capybara-webkit. This is quite essential for my gem. :-( Today there is no other (ruby-)alternative available:

If you need a tester for your new gem (version), I'm happy to add "a new driver" for that to proxy_rb's code base and try to run my test suite - Just send me a ping on Github. Is there a branch/repo already available? Unfortunately I'm not familiar with C++ in general and QT in particular, so I can't help you as a developer.

Cheers, Dennis