Closed piranout closed 11 years ago
This is something that .NET manages. You can set the maximum number of connections through your configuration. http://stackoverflow.com/questions/1361771/max-number-of-concurrent-httpwebrequests Op 16 okt. 2013 00:59 schreef "piranout" notifications@github.com:
I'm trying to use SimpleBrowser as a load testing client, and it's practically ideal for this. However, I can't find away around the nearly linear performance degradation with the instance count. I did notice that no matter how many instances I spin up, TCP connections max out at 2. Is there a way to increase the connections, or some other trick for getting better parallelism?
— Reply to this email directly or view it on GitHubhttps://github.com/Teun/SimpleBrowser.WebDriver/issues/18 .
Thanks; that's what I thought, though other users NB: I had the following in my app.config file, and was still only spawning 2 TCP connections per instance...
<system.serviceModel>
<bindings>
<netTcpBinding>
<binding maxConnections="1000"></binding>
</netTcpBinding>
<customBinding>
<binding>
<tcpTransport>
<connectionPoolSettings maxOutboundConnectionsPerEndpoint="1000"/>
</tcpTransport>
</binding>
</customBinding>
</bindings>
</system.serviceModel>
...but I still had to add the following to my thread spawner to get a connection per browser instance:
System.Net.ServicePointManager.DefaultConnectionLimit = 1000;
Hmm... do you have any insight into why I could still be seeing linear relationships between the number of concurrent browser instances & the response time per request? I instantiate the browsers & make a request to the domain I'm going to be hitting before measuring any timings, just to make sure I'm not counting initialization overhead. Then I loop through the instances & spawn a task for each that records request/response durations. Even with a TCP connection per instance, here's the performance curve I'm seeing:
Users:Median request time
========================
1 00:00:00.3098803
5 00:00:01.5973835
10 00:00:04.2335169
Is there anything static that would cause inter-instance contention? I think I'm approaching this right & not measuring extrinsic time, but maybe I'm overlooking something. Here's the code:
int[] userCounts = {1, 5, 10};
ResultCollector collector = new ResultCollector();
foreach (var userCount in userCounts)
{
List<TestRunner> testClassInstances = new List<TestRunner>();
for (int i = 0; i < userCount; i++)
{
TestRunner tester = null;
testClassInstances.Add(tester = new TestRunner());
// make a request to the home page of the domain being tested:
tester.Init();
}
List<Task> tasks = new List<Task>();
foreach (var instance in testClassInstances)
{
//capture outer variables:
TestRunner currentTestRunner = instance;
int concurrentUsers = userCount;
//spawn a task to
var task = Task.Factory.StartNew(
() =>
{ // run several requests to avoid measuring outliers:
for (int i = 0; i < 25; i++)
{
// perform an action that posts a form & checks for an
// element in the response; this action internally records
// its duration & stores it in the testResult returned:
var loginTestResult = currentTestRunner.LoginAsUser();
// store a memo of the current user count for this test:
loginTestResult.ConcurrentUsers = concurrentUsers;
// add the result to a list so we can summarize later:
collector.AddResult(loginAsUserResult);
}
});
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
}
// calculate the median time for each distinct concurrent user count:
collector.Summarize();
In the stackoverflow question, I posted earlier, Jon Skeet suggest reading this: http://msdn.microsoft.com/en-us/library/fb6y0fyc.aspx It is not the same thing that you have been playing with. This is about throttling the number of concurrent HTTP requests per domain. I still think that may help.
2013/10/16 piranout notifications@github.com
Hmm... do you have any insight into why I could still be seeing linear relationships between the number of concurrent browser instances & the response time per request? I instantiate the browsers & make a request to the domain I'm going to be hitting before measuring any timings, just to make sure I'm not counting initialization overhead. Then I loop through the instances & spawn a task for each that records request/response durations. Even with a TCP connection per instance, here's the performance curve I'm seeing:
Users:Median request time
1 00:00:00.3098803 5 00:00:01.5973835 10 00:00:04.2335169
Is there anything static that would cause inter-instance contention? I think\ I'm approaching this right & not measuring extrinsic time, but maybe I'm overlooking something. Here's the code:
int[] userCounts = {1, 5, 10};ResultCollector collector = new ResultCollector();foreach (var userCount in userCounts){ List
testClassInstances = new List (); for (int i = 0; i < userCount; i++) { TestRunner tester = null; testClassInstances.Add(tester = new TestRunner()); // make a request to the home page of the domain being tested: tester.Init(); } List tasks = new List (); foreach (var instance in testClassInstances) { //capture outer variables: TestRunner currentTestRunner = instance; int concurrentUsers = userCount; //spawn a task to var task = Task.Factory.StartNew( () => { // run several requests to avoid measuring outliers: for (int i = 0; i < 25; i++) { // perform an action that posts a form & checks for an // element in the response; this action internally records // its duration & stores it in the testResult returned: var loginTestResult = currentTestRunner.LoginAsUser(); // store a memo of the current user count for this test: loginTestResult.ConcurrentUsers = concurrentUsers; // add the result to a list so we can summarize later: collector.AddResult(loginAsUserResult); } }); tasks.Add(task); } Task.WaitAll(tasks.ToArray());}// calculate the median time for each distinct concurrent user count: collector.Summarize(); — Reply to this email directly or view it on GitHubhttps://github.com/Teun/SimpleBrowser.WebDriver/issues/18#issuecomment-26421022 .
Sorry, I thought this couldn't be the case, but the server I was hitting actually was degrading that linearly just going from one to five to ten users! Adding the line System.Net.ServicePointManager.DefaultConnectionLimit = 1000;
did increase the TCP connections, which I could verify in Process Explorer. When I hit a domain with more horsepower in its servers, I get flat response times scaling from 1 to 100 users, so I think SimpleBrowser.WebDriver is absolved!
Thanks for bearing with me! Cheers...
OK, cool, so then you actually found an issue using your test setup. That's what we all do this for, right?
I'm trying to use SimpleBrowser as a load testing client, and it's practically ideal for this. However, I can't find away around the nearly linear performance degradation with the instance count. I did notice that no matter how many instances I spin up, TCP connections max out at 2. Is there a way to increase the connections, or some other trick for getting better parallelism?