webmetrics / browsermob-proxy

NOTICE: this project has been forked and is being maintained at https://github.com/lightbody/browsermob-proxy
https://github.com/lightbody/browsermob-proxy
Apache License 2.0
233 stars 780 forks source link

SocketTimeoutException when getting POST parameters inside RequestInterceptor #114

Closed controllerface closed 11 years ago

controllerface commented 11 years ago

I've been using Browsermob-proxy and Selenium Webdriver for a few months now to run some simple tests and get more acquainted with browser automation testing and performance data gathering.

recently, i have started to experiment with the RequestInterceptor. I've created my own class that implements it, and everything seems to work fine. However, i have noticed that if I ever attempt to even READ what the post parameters are, it causes my browser test to stop, and FireFox (which i use primarily for testing) will display the "Oops!, Something went wrong. Firefox can't seem to load the page for some reason" message in the browser and in my console output window, i see an exception (java.net.SocketTimeoutException) thrown from the browsermob process.

It doesn't seem to matter what the POST data contains, but the test i've been using lately is on www.walmart.com. I have a very basic script that just goes to www.walmart.com, clicks on the sign in link, signs in, and then verifies that the user name is on the screen. This actually works flawlessly pretty much 100% of the time by itself, but as soon as I try and inspect the POST request that actually performs this login procedure, i see this error. It doesn;t matter if teh username and password are valid or not either, it simply seems to corrupt the request just by trying to get what the value of a parameter is.

Here is the entire contents of my request interceptor code:

import net.lightbody.bmp.proxy.http.BrowserMobHttpRequest;
import net.lightbody.bmp.proxy.http.RequestInterceptor;

public class ProxyRequestInterceptor implements RequestInterceptor
{
    public void process(BrowserMobHttpRequest request)
    {
        String testParam;
        testParam = request.getProxyRequest().getParameter("userName");

        System.out.println("Intercepted Request to: " + request.getProxyRequest().getRequestLine());
        if (testParam != null) System.out.println("TestParam: " + testParam );
    }
}

When I run my test with this code as is, going to walmart.com and logging in, when the test script gets to the point where it actually initiates the login request, I will actually see the parameter printed to the console, just as I am trying to do here...but then the socket timeout happens.

During my test, I am capturing both headers and content in a Har and dumping the entire thing to a text file every time I run the test. What I found was that on this POST request, the status code that comes back is -998 and in the "postData" object for this request, under "params" is just an empty "name": entry, and all of the actual POST data is missing.

Am I doing something wrong? In the future, I would really like to be able to not only capture this data, but also modify it as well, changing values like this before they are sent along to the server. I also have a ResponseInterceptor in this project, and I find that there I can see these values with no issue, but I'd really like to be able to see them (and possibly modify them) on the way out as well.

lightbody commented 11 years ago

NOTICE: this project has been forked and is being maintained at https://github.com/lightbody/browsermob-proxy

Can you please re-open this there? Ideally also submit a pull request with this change and I'll get it in faster. Sorry, wish I had a better answer for you right now, but my second daughter is due any day so time is tight :)

As for this specific issue: it is a known problem. Basically, when your interceptor reads the HTTP inputstream, it's reading a one-time use stream so the actual proxying breaks down. What we need to do is put a wrapper in place so that if you read it it captures the original in a buffer that can be replayed when doing the proxying. Hope that helps explain the issue.