Extremely slow proxing pages with 302 http status code

[miherlosev]

Open http://kinozal.tv/ in playground. Open http://kinozal.tv/ without proxing.

With proxing page will loaded after 45 sec. Without proxing - about 100 ms.

See video - http://screencast.com/t/BfrixaXWUM

[miherlosev]

I'm working on this

[miherlosev]

If destination request has 301, 302 status code we should not to perfom processing strategy calculation, fetching destination body, processing it, etc. We need to send responce headers and finish request.

[churkin]

@miherlosev Pay attention to:

https://tools.ietf.org/html/rfc2616#section-10.3.2

The new permanent URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).

[miherlosev]

use test cases from: https://github.com/request/request/blob/master/tests/test-redirect-auth.js https://github.com/request/request/blob/master/tests/test-redirect-complex.js https://github.com/request/request/blob/master/tests/test-redirect-auth.js

[miherlosev]

Also for another sites: 1)http://www.wp.pl - with proxy - 27 sec, without - 2.5 sec 2)http://www.y8.com - with proxy - 5 sec, without - 2 sec

[LavrovArtem]

I'm working on this

[LavrovArtem]

When the browser receives status 302 and the location header, it immediately sends a new request without waiting for the old request to complete. It takes http://kinozal.tv/ 46 seconds to complete its response. Since the headers arrive immediately, the browser starts loading a url from the location header at that moment. An issue with Hammerhead is that headers are sent only after the body is fetched and processed, so that the page is being loaded for 46 seconds while the redirect happens after that. The problem can be solved in two ways:

1. Check if the received status implies redirect (301, 302, 303, 307) and the location header is present. If so, end the request and send the processed headers.
2. Send the processed headers immediately after receiving them from the destination resource. But here it comes an issue with the content-length header value - we can determine it only after the resource body is processed.

\cc @churkin @inikulin

[inikulin]

@LavrovArtem What's pros and cons of each approach? Is there any sufficient cons for each approach that don't allow you to decide yourself?

[LavrovArtem]

@inikulin For first way:

• absence cons of the second way
• we decide whether to continue to load the response body or not

For second way:

• browser independently decides when to start his redirect
• issue with the content-length header value - we can determine it only after the resource body is processed

I think that second way is better, but i don't know how solve issue with content-length

[inikulin]

I think that second way is better, but i don't know how solve issue with content-length

Me neither

[churkin]

So we choose the first option

[lock[bot]]

This thread has been automatically locked since it is closed and there has not been any recent activity. Please open a new issue for related bugs or feature requests. We recommend you ask TestCafe API, usage and configuration inquiries on StackOverflow.