ollie314 / browserscope

Automatically exported from code.google.com/p/browserscope
Apache License 2.0
0 stars 0 forks source link

Max connection test is broken #310

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?
1. Run the "Max connections" test in Chrome (I'm using 13.0.777.1, but I don't 
think that particularly matters).
2. Watch the images that load. Notice that several of them are broken.

What is the expected output? What do you see instead?
All images should load a 200.

Please provide any additional information below.
It looks like the server has a maximum number of connections and returns 500s 
for images once that limit is reached. The 500s break the test for two reasons:
1. They don't have a 6s server delay.
2. They fire onerror instead of onload so the countImage() function is never 
triggered.

This means that Chrome's level of parallelism is actually overloading the 
server causing the test to score lower (because the 500s aren't counted). The 
chrome networking folks have informed me that the actual global socket limit is 
256, not 35 as reported by browserscope.

Original issue reported on code.google.com by tonyg@google.com on 27 May 2011 at 8:20

GoogleCodeExporter commented 9 years ago
This has been fixed by changing the hosted version of resource.cgi to the 
latest version of App Engine instead of Media Temple. The new App Engine 
python2.7 backends can support the kind of concurrency we need for this test.

I don't know how long the server quota will hold up, but I get 60 now with 
Chrome =)

Original comment by els...@gmail.com on 29 Dec 2011 at 11:13

GoogleCodeExporter commented 9 years ago
If it is reporting 60, the test still isn't working correctly :(

Chrome holds open a max of 256 sockets. You can view them by opening 
chrome://net-internals/#sockets in one tab and then browsing around to some 
heavy sites in other tabs. Watch the "transport_socket_pool" line summary and 
the socket details below. That line lists a max of 256, and it isn't too hard 
to get it up that limit by opening several news sites quickly.

Original comment by tonygent...@gmail.com on 30 Dec 2011 at 11:07

GoogleCodeExporter commented 9 years ago
60 is the max that get tried for in that test as it is written/designed. I'll 
test today to see how much it can handle concurrently just to see if we could 
possibly get a more accurate number there.

Original comment by els...@gmail.com on 30 Dec 2011 at 3:22

GoogleCodeExporter commented 9 years ago
Ah, I see that in the description of the test now. Makes sense since it is 
documented. Thanks.

Original comment by tonyg@google.com on 30 Dec 2011 at 3:30

GoogleCodeExporter commented 9 years ago
We could try finding the max, but it would be very time consuming. Many 
browsers (esp. mobile and behind proxies) support very few max connections, so 
we don't want to do 256 out of the gate. We could probably get a good estimate 
with 3 iterations, but that'll add 30-120 seconds to the test suite, which is 
already very long.

Original comment by stevesou...@gmail.com on 4 Jan 2012 at 5:38

GoogleCodeExporter commented 9 years ago
To allow for probing across a wider dynamic range of connection limits, is 
there any way we could have this test progressively increase the limit, so when 
it finds the limit is the same between two iterations, it stops increasing it?

Original comment by m...@google.com on 29 May 2012 at 5:46

GoogleCodeExporter commented 9 years ago
It's possible but not practical. The network test suite takes too long as it is 
- this would dramatically increase the time to run the test suite.

Original comment by stevesou...@gmail.com on 1 Jun 2012 at 11:22