WebDriver lacks HTTP response header and status code methods

[lukeis]

Originally reported on Google Code with ID 141

It would be useful to have methods to read the HTTP status code and headers from an
HTTP 
response.  HtmlUnit already has this functionality, but it's not been brought through
to WebDriver.

Reported by stephendv on 2009-02-01 16:05:44

[lukeis]

another vote for reopening.

Http Status Codes, etc. should at lease be an accessor, that way we can assert and
verify it...

Reported by qv.personal on 2012-01-11 23:47:55

[lukeis]

vote for re-opening

Reported by kromped on 2012-02-18 03:57:33

[lukeis]

+1 this is a really important missing feature, please return status code from driver.get

Reported by tom.medhurst on 2012-02-21 12:45:04

[lukeis]

I'm troubleshooting some weird problem with the server starting to return 404 on occasion.
It would be really helpful to get early test failings, but alas :(

Please, consider reopening

Reported by dkroot2 on 2012-02-23 15:20:19

[lukeis]

This really would be useful. Even if it is just for 404 and 500's

Reported by devaid on 2012-03-15 12:37:09

[lukeis]

also voting for that feature! would need at least the info, if the page could be opened
correctly or not.

Reported by klemensl on 2012-03-21 12:03:21

[lukeis]

please consider reopening

Reported by a.ratto@stormreply.com on 2012-03-23 15:31:51

[lukeis]

yea, should be re opened

Reported by sharma.dharmpal on 2012-03-29 18:09:20

[lukeis]

I don't understand the refusal for this feature either. There are a lot of potential
uses, like checking caching or language settings. Not all tests can easily be done
with a simple http client. Cache invalidation in particular is something I want to
test with complex scenarii, which is exactly what webdriver is designed for.

Reported by arno@arnoo.net on 2012-04-04 08:54:47

[lukeis]

Please, please, please fix this. The reasons given not to are complete nonsense. Sometimes
you need to verify stuff that isn't directly in the DOM, like cache stuff, status codes,
and other HTTP headers. Sure, you can do this by making the request without WebDriver,
but when you are simulating a user flow you can't just make two separate requests and
assume that it won't break anything. What's more, that would just be ugly. Here at
matchFWD we need this feature for various test cases. +1

Reported by fletcher@matchfwd.com on 2012-04-05 18:13:25

[lukeis]

This data can be useful.  Using a proxy, e.g. the BrowserMob proxy is an easy way to
get this data, without any of the drawbacks mentioned.

Reported by dawagner on 2012-04-05 22:40:09

[lukeis]

#Use grab lib on python:

from grab import Grab, GrabError
from urllib import quote
import re

g = Grab()
g.go(self.base_url)
print g.response.code

Reported by bo858585 on 2012-04-06 10:59:51

[lukeis]

Another plea to add status code support.  It's really frustrating not having access
to it.  Can totally appreciate not exposing headers, but having to chase for text for
error pages, not found, etc...  is verbose and brittle.  Setting up a proxy just for
status code?  Is there technical difficultly involved in exposing status code, because
if not, from a user's perspective it's hard to understand the resistance to what many
feel is a pretty reasonable thing!

Reported by orutherfurd on 2012-04-12 12:38:07

[lukeis]

Another vote for adding this simple, reasonable feature.  

Reported by walker.sf on 2012-05-18 20:07:18

[lukeis]

What drawbacks are there? Nobody wants the extra headache of running and maintaining
proxy code on top of their already complex server/browser testing code. Look, it's
just a single integer that needs to be exposed on the browser object. That's all we
need. It couldn't be simpler.

Reported by fletcher@matchfwd.com on 2012-05-18 20:14:03

[lukeis]

I hesitate to proverbially stick my head in the lion's mouth, but it's not as simple
as adding "a single integer that needs to be exposed on the browser object" in the
WebDriver API. Leaving aside the need to define exactly what is meant by "status code",
are you willing to live with an API that doesn't work for every browser? Think carefully
before you blithely answer, "Yes." There is at least one browser (IE) that does not
provide any mechanism whatsoever to expose the information you're asking for to the
user, and, though I've not actually researched it to be able to say with certainty,
I strongly suspect that there are others that don't expose it as well (Safari, desktop
and mobile, come to mind).

I hope we can all agree that an API that is inconsistent across browsers is vastly
inferior to a solution that works correctly in all browsers. The reason WebDriver won't
provide it is not merely aesthetic; it's also technical, in that not all browsers provide
the information in a way that a library like WebDriver can access. A proxy solution,
on the other hand, *will* work for every browser, and can provide you with status codes,
HTTP headers, and much more information.

Reported by james.h.evans.jr on 2012-05-20 15:51:56

[lukeis]

Thanks James

Reported by etudiantemiage on 2012-06-20 14:51:24

[lukeis]

Well, I think we are already living with an api that doesn't work for every browser.
And that is one of the reasons why we have the desired capabilities settings, right?

Reported by bielmooca on 2012-06-22 03:19:49

[lukeis]

Well, for our projects being able to test status code in Firefox would meet our needs.
When we automate running our tests against multiple browsers we can just skip that
test for browsers that don't support it. I appreciate the desire to maintain a fully
cross-browser api, but where that means "lowest common denominator" features and missing
out functionality that is so obviously desired by a large number of users it would
seem that sometimes practicality can trump purity.

Reported by fuzzyman on 2012-06-22 18:23:08

[lukeis]

Another 'vote' for implementing a 'get the last status code' method.

The excuses from the project team so far don't really hold water. The status code *is*
exposed to the user in at least IE - if i serve the page <p>Hello!</p> with a 200,
it will be displayed, but with an error code, the user will see the generic IE error
page instead.

Redirects were mentioned as a problematic case. In keeping with the idea that the API
should reflect the user experience, i would suggest that redirects should be followed
as usual, and only the status code of the final page load be exposed. An option to
not follow redirects, which would then leave the redirect status exposed, would be
useful, but not essential.

To address the matter of consistency, i'd say the preference is: API which works consistently
across all platforms > API which works consistently across some platforms, and fails
loudly on others > API which works inconsistently across platforms > no API at all.
Currently, we are in that last situation.

Reported by tomwhoiscontrary on 2012-06-23 13:06:28

[lukeis]

Thanks Tom, for putting that so well.

Reported by fletcher@tomalty.com on 2012-06-23 14:21:57

[lukeis]

Another vote for adding this useful feature.  

Reported by i.sammie on 2012-07-25 01:39:28

[lukeis]

At the risk of adding yet another "me too" post, I would like to express my opinion
that this is a very important missing feature. I recently modified some tests that
used the Twill package to be able to use WebDriver and found fairly straightforward
ways to accomplish everything the tests had been doing save checking the HTTP response
code.

While I can accept that there may be serious technical limitations to providing this
feature for some browsers, the attitude that it's not needed seems misguided at best.
Explanations for why it's difficult to solve a problem or requests for help are acceptable,
but simply denying the problem's existence is not. I'd far prefer to have the feature
available in only some browsers than in none.

Reported by JonathanRRogers on 2012-07-25 01:50:41

[lukeis]

Once again: use external proxy software, e.g. browsermob proxy.

Here is a sample of code that uses browsermob proxy to save last response code to a
variable:

    int lastResponseCode;

    ProxyServer server = new ProxyServer(8888);
    server.start();

    server.addResponseInterceptor(new HttpResponseInterceptor() {
      @Override
      public void process(HttpResponse response, HttpContext context) throws HttpException,
IOException {
        lastResponseCode = response.getStatusLine().getStatusCode();
      }
    });

    DesiredCapabilities caps = new DesiredCapabilities();
    caps.setCapability(CapabilityType.PROXY, server.getProxy());
    WebDriver driver = new FirefoxDriver(caps);
    ...

Reported by barancev on 2012-07-25 09:34:04

[lukeis]

Sorry, in the code above there should be server.seleniumProxy() instead of server.getProxy()

Reported by barancev on 2012-07-25 09:35:12

[lukeis]

So, barancev, how would I use BrowserMob to get at an HTTP response in Python? How about
PHP, Perl, Ruby or C#? The idea that this feature is missing because of a desire for
a consistent API and the fact that you can use a different mechanism to work around
the shortcoming falls pretty flat if the workaround only applies to one of the six
languages supported by Selenium.

Reported by JonathanRRogers on 2012-07-25 15:38:37

[lukeis]

Even in Java, the BrowserMob solution is far from optimal: the code snippet listed above
doesn't work for localhost connections due to https://github.com/webmetrics/browsermob-proxy/issues/55

Aside from that: why on earth should one need to bother starting a complete server
and worry about proxy port selection (in the event of concurrent test runs on the same
machine) while all that one's interested in is data that is returned as part of the
GET request *anyway*?

Reported by stephan202 on 2012-07-25 15:48:19

[lukeis]

One should bother to do that because otherwise the Selenium committers have
to bother with it--in all permutations, creating new dependencies, new
external library interactions, and all sorts of complication.

As an early supporter of this feature, I understand your pain, but I'm now
converted--it's *because* Selenium is such a well-used, strongly supported
platform that we expect it to be all things to all people. The truth is,
Selenium is a robot arm, using what amounts to exploits to click buttons
and send commands to a gigantic thing called a web browser. Selenium is
about a million miles from the HTTP request/response cycle, and the design
from the beginning has been to keep it that way. The browsers do not supply
this information by default in any consumable form, and as far as I know
they never plan to.

The best hope we have is that someone out there in the open source
community can create a "standard" plug-in that works similarly in all
browsers, then provide an API that you can talk to in order to harvest this
information. That's exactly what the selenium implementers would have to do
in order to make this happen, and since it doesn't follow with the design
goals, they don't plan to do it. If you want to have a go at it, just to
find out how big a task it actually is, start with Chrome (arguably the
easiest one) and see what kind of effort it requires. Decide on the
interface you'd need to provide, then code to that. Then do the same for
Firefox. If you're still into it, then create it for IE. Then maintain it
for a while, making sure the API always works with the current version of
Selenium and the current version of every browser. Even then, however, it
would be a *separate tool*.

Imagine I have a remote control car. I push buttons and the car goes
forward, backward, left, right. That leads me to the conclusion that it
would be easy to just add a window into the gas consumption at different
speeds, so I can track how much money it costs to run the car. In truth,
what seems like a cohesive control is really 4 independent circuits,
sending 4 separate signals to an antenna--and they're all dumb as dirt. The
remote control doesn't even know there's a motor involved. It would take a
complete redesign of the most basic concepts of the remote control to
understand how to handle that kind of feedback, and it just wasn't part of
the original goals of the system.

The analogy may not be perfect, but you get the point. This is a separate
concept, and needs to be implemented separately. If we can unify an
interface around it, it will be easy to create a bridge to Selenium, but it
can't be a part of the core. The core knows *nothing* about HTTP.
MM

Reported by mmerrell on 2012-07-25 16:30:36

[lukeis]

Great explanation. Thanks MM!

Reported by x.bgoad@reverbnation.com on 2012-07-25 17:24:26

[lukeis]

@MM: I appreciate your taking time to explain your perspective.

I think a big part of the problem here is a strongly differing understanding, or confusion,
about the "design from the beginning" and "the original goals of the system". You write
about "Selenium," but when you talk about browser automation you're describing (Selenium)
WebDriver. Selenium (and even Selenium WebDriver) used to have a self-described design
goal of being suitable for web app testing (see comment 39). Even now, the FAQ states
that WebDriver is a "tool for writing automated tests of web sites", not merely for
driving the browser. Another FAQ states "the aim is the same (to allow you to test
your web app)" (https://code.google.com/p/selenium/wiki/FrequentlyAskedQuestions#Q:_So,_is_it_like_?_Or_?).
A large user base has invested in that function. Testing response status codes *is*
an important part of the goal of web app testing, even if it is not part of the goal
of "browser automation".

I'm open to the argument that this feature would be difficult to implement consistently
across all platforms in the way that WebDriver works. Of course this leaves open other
options, such as implementing it for some browsers.

What I find objectionable is a statement that this sort of function was never part
of Selenium's design goals. See the above statements from Selenium's own sources.

I think the RC car example is very revealing, regarding the contrast between how Selenium
is described by opponents of this issue, and how it has historically and publicly been
described. An RC car remote is not designed for testing anything, nor is it described
as being suitable for that purpose. Of course it would require a complete redesign
to start retrieving and displaying data, because such a purpose is far from its original
design goals. In contrast, go to the FAQ page for Selenium and search for occurrences
of the word "test". Then ask whether "automation" really covers the described goals
of Selenium. Selenium is not described merely as a remote control for browsers, but
as a tool for testing web sites. Wikipedia's article on Selenium opens, "Selenium is
a portable software testing framework for web applications." If Selenium is not for
web app testing, the project has a significant public perception problem to overcome.

If Selenium stopped being a tool for testing web sites, you might then be able to argue
as you have, that exposing status codes is not an important part of its goals. But
that would be a sad day for open source web app test automation, and a disappointment
to the many users who signed on to its *original* goals.

If it's true that negotiations are underway with W3C to make WebDriver an internet
standard, maybe that should be used as an opportunity to specify a uniform API for
browsers to expose response status information. It wouldn't have to be mandatory, but
it would be an opportunity for browser vendors to increase interoperability, and to
improve the quality of web applications by facilitating automated testing.

Reported by huttarl on 2012-07-25 19:15:06

[lukeis]

There are already plenty of things you can do with WebDriver that are nothing like the
way an ordinary user uses a browser, such as find form elements using ids or XPath.
Users can't read text that is the same color as its background but WebDriver can. If
WebDriver were really limited to interacting with a browser in the exactly the same
way as an ordinary user, it would have to operate on the image presented in the window
using only key presses and mouse movements and clicks. Not only would this be vastly
more complex and difficult than the way it does work and would make writing tests far
more difficult, but it would obviously disqualify HTMLUnit, which generates no image
at all.

To say that "The core knows *nothing* about HTTP." is about as meaningful as saying
"The core knows *nothing* about GUI toolkits or window systems." Both statements are
true but obviously, WebDriver is able to interact with browsers that do have GUI widgets.
One thing even more universal than a browser's interaction with a window system (HTMLUnit
doesn't do that) is the fact that a browser interacts with HTTP and must understand
HTTP responses. Therefore, a consistent WebDriver API for exposing a browser's knowledge
of such responses can be designed.

I understand that implementing such an API for specific browsers has technical limitations
and therefore may be very difficult. However, that is a totally different issue from
the main reason given for removing this very useful feature from Selenium. As several
others have already pointed out, it's inconsistent to claim that HTTP response information
is outside of the scope of the project while page source and the browser's cookie store
are not. There are also varying levels of support for existing parts of the API in
the several browsers on the several platforms so the attitude that if a feature can't
be implemented completely on all browsers, it must not be implemented on any of them
is manifestly silly.

Reported by JonathanRRogers on 2012-07-25 19:24:06

[lukeis]

I agree with the comments above: not exposing the HTTP response headers is just silly.
 It won't pollute the API, and if you're *really* that concerned about it, add it in
an implicitly-implemented interface, like ITakeScreenshots. 

Reported by mbhoneycutt on 2012-08-21 16:35:27

[lukeis]

For those looking for a workaround, try putting your response code into your html output.
I'm putting in the head section like this:

  <meta name="response" value="403" id="web_403"/>

And the findElement(By.id("web_403"));

Reported by rogerkeays.RK on 2012-10-24 11:04:29

[lukeis]

We can also use js to get response code ,

Reported by manasatr on 2012-10-24 13:17:37

[lukeis]

Thanks #88! very useful :) I used $_SERVER['REDIRECT_STATUS'] but if you're on php5.4
or above you could use http_response_code(). It's definitely a workaround though.

I'm also pretty annoyed that we have to do this just to find out whether the current
page is a 404 or whatever.

Current reasoning for "wont fix" definitely comes across to an outsider as an excuse
to throw this in the "too hard" basket rather than a real idealogical argument.

Reported by thedavidmeister on 2012-10-30 02:59:37

[lukeis]

This is a too idealistic view.
And the third sentence at seleniumhq.org: says "Primarily it is for automating web
applications for testing purposes, [...]"
And I can view the HTTP status code with my plain Firefox (CTRL + i) or plain Chrome
(SHIFT + CTRL + i) anyway.

I don't get it

Reported by irae.hueck.costa on 2012-11-14 17:38:56

[lukeis]

Sigh. My last-ditch, for-realsies final attempt to inject some rationality into the
inflammatory rhetoric surrounding this issue: http://jimevansmusic.blogspot.com/2012/07/webdriver-y-u-no-have-http-status-codes.html

Reported by james.h.evans.jr on 2012-11-15 14:52:30

[lukeis]

Interesting blog Jim. For the rest of us who have work to do I posted an article on
workarounds and alternative solutions to this problem: http://www.ninthavenue.com.au/how-to-get-the-http-status-code-in-selenium-webdriver

Reported by rogerkeays.RK on 2012-11-15 23:06:57

[lukeis]

Thank you Roger.

Reported by huttarl on 2012-11-16 03:22:00

[lukeis]

does htmlunitdriver have ; getServerResponse() ,getServerResponseCode() ,getResponseHeaders()
,getInputStream()  yet and when will it be available ?

Reported by kakuffo@curoonline.com on 2013-01-17 03:30:29

[lukeis]

Another vote for HTTP headers.  And I don't take kindly to the condecension.  

Reported by brianbegy on 2013-02-11 17:17:14

[lukeis]

@96

It really isn't.

Instead of saying "another vote" and "don't be condescending", how about you get your
fingers out and code the solution?

Reported by arran.huxtable on 2013-03-05 13:19:55

[lukeis]

@97: There doesn't need to be an "instead of". Brian's entitled to a "vote" as much
as anyone else.

Reported by huttarl on 2013-03-05 14:14:12

[lukeis]

@98, not on this issue, reading Jim's blog post above, one of the suggested solutions
is get coding yourself! The other solution is using the right tool.

It doesn't sound like it'll get done by one of the Selenium dev's and one of them has
said there are other tools out there that you can do this with! So why needlessly say
"oh yeah, I'd like this too" and the added "don't be condescending" - *that* is unnecessary.

Reported by arran.huxtable on 2013-03-05 14:42:49

[lukeis]

@99: Nonsense. The point here is to have the feature *inside* WebDriver. Unless one
is willing to maintain a fork, there is no point in coding anything if the maintainers
are unwilling to extend the API in the proposed manner.

Reported by stephan202 on 2013-03-05 14:55:48

[lukeis]

I'm currently implementing tests to ensure links we have to external websites open.
Obviously it is our problem if our links do not load for the user, but then under the
current scope of WebDriver I essentially have to check for text or an element on the
external site.

Ensuring the content of an external site shouldn't then be my concern, I simply wish
to know that the user would be able to load the site. Doesn't this kind of usecase
provide a positive argument for the implementation of status codes in WebDriver?

Reported by theshow on 2013-04-11 07:12:54

[lukeis]

This bug talks about two separate features - set header on request, and get header on
response. We shouldn't mix them together.

We really need the "setter" feature to test Wikipedia's mobile responses for different
mobile carriers (see Wikipedia Zero). To test, we spoof the X-Forward-For header with
a fake IP addresses, after which it goes through all the caching proxies (varnish),
and apache servers checking that responses are appropriate of the specific region/carrier.
For now we are forced to emulate it with a URL parameter, which is ignored by varnish,
hence making the tests much less useful.

The getter functionality is "nice to have", but not as important.

Reported by yastrakhan@wikimedia.org on 2013-05-03 17:56:21

[lukeis]

To be honest, most of the discussion I've seen above is pointless.

Fact: a "web testing tool" that does not handle response codes at all is simply defective.
No excuses, no BS. The excuses paraded here by the devs have been incredibly lame.

(1) "Emulating user experience" is not an excuse because USERS CAN SEE when there is
a 404 or 500, and it has a drastic effect on their subsequent behavior. The fact that
Selenium-Webdriver cannot deal with this information at all is a severe deficiency,
because it therefore FAILS to emulate the user experience in a VERY big way.

(2) Technical issues are IRRELEVANT. The feature needs to be there (see point 1), even
if it isn't 100% consistent.

(3) External tools are also irrelevant. If Selenium-Webdriver does not have this feature,
it is defective (see point 1 again). If an external tool is necessary to get this information,
then it needs to be incorporated into Selenium-Webdriver in order to truly emulate
the "user experience". YOUR MOST IMPORTANT "USERS" ARE DEVELOPERS. Mom and Pop at home
don't use Selenium-Webdriver!

It all falls back to point 1. Selenium devs say they want to emulate the user's experience.
If they can't deal with HTTP responses INSIDE Webdriver, they have no chance in hell
of fulfilling that goal. Period. End of story.

I shall not apologize for being blunt. I feel this needed to be said.

Reported by lonny6 on 2013-06-20 18:39:50

[lukeis]

(1) User doesn't see 404 or 500, they see how it's rendered on the web page and react
accordingly.  WebDriver can do that too.

(2) Handling raw response codes has been declared out of scope - this feature will
NOT be implemented.

Reported by jmleyba on 2013-06-20 19:05:24

[lukeis]

and I quote:

"YOUR MOST IMPORTANT "USERS" ARE DEVELOPERS".  Lonny, you're right.  As a developer,
I suggest that you take some time on an open source project, and contribute your solution
to this issue!  The best way to make SE WebDriver better is to contribute your own
solution.  As many of us always say, PATCHES WELCOME!

Reported by llaskin on 2013-06-20 20:17:39