Increase parallel TCP connections?

[piranout]

I'm trying to use SimpleBrowser as a load testing client, and it's practically ideal for this. However, I can't find away around the nearly linear performance degradation with the instance count. I did notice that no matter how many instances I spin up, TCP connections max out at 2. Is there a way to increase the connections, or some other trick for getting better parallelism?

[Teun]

This is something that .NET manages. You can set the maximum number of connections through your configuration. http://stackoverflow.com/questions/1361771/max-number-of-concurrent-httpwebrequests Op 16 okt. 2013 00:59 schreef "piranout" notifications@github.com:

I'm trying to use SimpleBrowser as a load testing client, and it's practically ideal for this. However, I can't find away around the nearly linear performance degradation with the instance count. I did notice that no matter how many instances I spin up, TCP connections max out at 2. Is there a way to increase the connections, or some other trick for getting better parallelism?

— Reply to this email directly or view it on GitHubhttps://github.com/Teun/SimpleBrowser.WebDriver/issues/18 .

[piranout]

Thanks; that's what I thought, though other users NB: I had the following in my app.config file, and was still only spawning 2 TCP connections per instance...

  <system.serviceModel>
    <bindings>
      <netTcpBinding>
        <binding maxConnections="1000"></binding>
      </netTcpBinding>
      <customBinding>
        <binding>
          <tcpTransport>
            <connectionPoolSettings maxOutboundConnectionsPerEndpoint="1000"/>
          </tcpTransport>
        </binding>
      </customBinding>
    </bindings>
  </system.serviceModel>

...but I still had to add the following to my thread spawner to get a connection per browser instance:

System.Net.ServicePointManager.DefaultConnectionLimit = 1000;

[piranout]

Hmm... do you have any insight into why I could still be seeing linear relationships between the number of concurrent browser instances & the response time per request? I instantiate the browsers & make a request to the domain I'm going to be hitting before measuring any timings, just to make sure I'm not counting initialization overhead. Then I loop through the instances & spawn a task for each that records request/response durations. Even with a TCP connection per instance, here's the performance curve I'm seeing:

Users:Median request time  
========================
1       00:00:00.3098803
5       00:00:01.5973835
10      00:00:04.2335169

Is there anything static that would cause inter-instance contention? I think I'm approaching this right & not measuring extrinsic time, but maybe I'm overlooking something. Here's the code:

int[] userCounts = {1, 5, 10};
ResultCollector collector = new ResultCollector();
foreach (var userCount in userCounts)
{
    List<TestRunner> testClassInstances = new List<TestRunner>();
    for (int i = 0; i < userCount; i++)
    {
        TestRunner tester = null;
        testClassInstances.Add(tester = new TestRunner());
        // make a request to the home page of the domain being tested:
        tester.Init();  
    }
    List<Task> tasks = new List<Task>();
    foreach (var instance in testClassInstances)
    {
        //capture outer variables:
        TestRunner currentTestRunner = instance;
        int concurrentUsers = userCount;
        //spawn a task to 
        var task = Task.Factory.StartNew(
            () =>
            {   // run several requests to avoid measuring outliers:
                for (int i = 0; i < 25; i++)
                {
                    // perform an action that posts a form & checks for an 
                    // element in the response; this action internally records
                    // its duration & stores it in the testResult returned:
                    var loginTestResult = currentTestRunner.LoginAsUser();
                    // store a memo of the current user count for this test:
                    loginTestResult.ConcurrentUsers = concurrentUsers;
                    // add the result to a list so we can summarize later:
                    collector.AddResult(loginAsUserResult);
                }
            });
        tasks.Add(task);
    }
    Task.WaitAll(tasks.ToArray());
}
// calculate the median time for each distinct concurrent user count: 
collector.Summarize();

[Teun]

In the stackoverflow question, I posted earlier, Jon Skeet suggest reading this: http://msdn.microsoft.com/en-us/library/fb6y0fyc.aspx It is not the same thing that you have been playing with. This is about throttling the number of concurrent HTTP requests per domain. I still think that may help.

2013/10/16 piranout notifications@github.com

Hmm... do you have any insight into why I could still be seeing linear relationships between the number of concurrent browser instances & the response time per request? I instantiate the browsers & make a request to the domain I'm going to be hitting before measuring any timings, just to make sure I'm not counting initialization overhead. Then I loop through the instances & spawn a task for each that records request/response durations. Even with a TCP connection per instance, here's the performance curve I'm seeing:

Users:Median request time

1 00:00:00.3098803 5 00:00:01.5973835 10 00:00:04.2335169

Is there anything static that would cause inter-instance contention? I think\ I'm approaching this right & not measuring extrinsic time, but maybe I'm overlooking something. Here's the code:

int[] userCounts = {1, 5, 10};ResultCollector collector = new ResultCollector();foreach (var userCount in userCounts){ List testClassInstances = new List(); for (int i = 0; i < userCount; i++) { TestRunner tester = null; testClassInstances.Add(tester = new TestRunner()); // make a request to the home page of the domain being tested: tester.Init(); } List tasks = new List(); foreach (var instance in testClassInstances) { //capture outer variables: TestRunner currentTestRunner = instance; int concurrentUsers = userCount; //spawn a task to var task = Task.Factory.StartNew( () => { // run several requests to avoid measuring outliers: for (int i = 0; i < 25; i++) { // perform an action that posts a form & checks for an // element in the response; this action internally records // its duration & stores it in the testResult returned: var loginTestResult = currentTestRunner.LoginAsUser(); // store a memo of the current user count for this test: loginTestResult.ConcurrentUsers = concurrentUsers; // add the result to a list so we can summarize later: collector.AddResult(loginAsUserResult); } }); tasks.Add(task); } Task.WaitAll(tasks.ToArray());}// calculate the median time for each distinct concurrent user count: collector.Summarize();

— Reply to this email directly or view it on GitHubhttps://github.com/Teun/SimpleBrowser.WebDriver/issues/18#issuecomment-26421022 .

[piranout]

Sorry, I thought this couldn't be the case, but the server I was hitting actually was degrading that linearly just going from one to five to ten users! Adding the line System.Net.ServicePointManager.DefaultConnectionLimit = 1000; did increase the TCP connections, which I could verify in Process Explorer. When I hit a domain with more horsepower in its servers, I get flat response times scaling from 1 to 100 users, so I think SimpleBrowser.WebDriver is absolved!

Thanks for bearing with me! Cheers...

[Teun]

OK, cool, so then you actually found an issue using your test setup. That's what we all do this for, right?