ClientPool - extending pool / add new clients dynamically

[RomanPoprava]

Currently given by NBomber client pool can be initialized with constant number of clients, before the bombing phase. It's hard to calculate necessary number of clients for open system type load simulation. So in some cases you can underestimate clients count needed for client pool. And during bombing your client pool will not have any available clients for the new threads.

Example: in WithInit() you are creating 10 clients and adding them into client pool. in .WithLoadSimulations() you are setting e.g. Inject for 20 virtual users with interval 1 second with duration 1 minute.

[AntyaDev]

Hi @RomanPoprava , The ClientPool suppose to be used for a Closed System model. Do you have examples of such client pools in other load-testing frameworks? If yes, could you please share the link? It will help us to re-design it properly.

[RomanPoprava]

Unfortunately I don't have any examples in existing load-testing frameworks. But I am going to explain what I did to gain dynamically extending client pool. So, basically my idea was to store used and unused clients separately. I have two thread-safe lists with used and unused clients regardless.

1. Before scenario starts - I am initiating client pool for 10 clients.
2. Each copy of the scenario is going to take ANY available client from unused clients list. Each copy of scenario has its own thread number. I am keeping that thread number. Client is going to be moved from unused list to used list.
3. When scenario finishes, I am going to release client by that thread number and client moves from used to unused.
4. If unused clients list is going to run out of the clients, then I am calculating next thread number and generating new client by adding it used list with that thread id.

This approach can be used for testing both Open and Closed systems.

And here is some example part of the implementation:

public ScenarioProps GenerateScenario()
{
    var pool = new CustomClientPool();

    return Scenario.Create("some name", async context =>
    {
        var client = pool.GetAvailableClient();

        //step 1
        //...
        //step N

        pool.ReleaseClient(client.Id); // client.Id is a threadNumber
    })
        .WithInit(context => pool.Init(clientCount: 100))
        .WithClean(context => pool.Clean());
}

And the client pool class is:

 public class CustomClientPool
 {
     #region Fields

     private readonly object _lock = new();

     #endregion

     #region Properties

     protected int GeneralClientsCount { get; set; }
     protected ConcurrentDictionary<int, MyClient> UsedClients { get; } = new();
     protected ConcurrentDictionary<int, MyClient> UnusedClients { get; } = new();

     #endregion

     #region Methods

     protected MyClient GenerateClient(int id)
     {
        //create your own client
        //in my case I used my own class for Client based on FlurlClient
     }

     public override void ReleaseClient(int threadNumber)
     {
         if (UsedClients.TryRemove(threadNumber, out var client))
         {
             //Clear client headers and cookies if needed

             UnusedClients.TryAdd(threadNumber, client);
         }
     }

     private int GetNextThreadNumber()
     {
         lock (_lock)
         {
             return GeneralClientsCount++;
         }
     }

     public void Dispose()
     {
        //dispose all elements within UsedClients
        //dispose all elements within UnusedClients
     }

     public void Init(int clientsCount)
     {
         GeneralClientsCount = clientsCount;

         for (int i = 0; i < clientsCount; i++)
         {
             UnusedClients.TryAdd(i, GenerateClient(i));
         }
     }

     public MyClient GetAvailableClient()
     {
         foreach (var kvp in UnusedClients)
         {
             if (UnusedClients.TryRemove(kvp.Key, out var client))
             {
                 UsedClients.TryAdd(kvp.Key, client);

                 return client;
             }
         }

         var newThreadNumber = GetNextThreadNumber();
         var newClient = GenerateClient(newThreadNumber);
         UsedClients.TryAdd(newThreadNumber, newClient);

         return newClient;
     }

     #endregion
 }

And my client is:

public class CookiesClient
{
    public IFlurlClient FlurlClient { get; }
    public int Id { get; }
}

I am actively testing and finishing implementation of this pool right now, so please let me know what do You think, maybe You have some better ideas how to handle this case. Thank you. :)

[AntyaDev]

@RomanPoprava It seems that you would like to have a dynamically Resizable Client Pool. I like your idea, and I remember I also wanted to provide something similar. The only thing that stopped me from building it was that such an abstraction is problematic to make generic. Imagine you have a WebSocket, and you would like to use a Resizable Client Pool. Imagine now that your ClientPool is full, and it needs to be resized to add more WebSocket clients. The thing is that adding WebSocket means not just creating a WebSocketClient (like you have with HTTP) but rather creating an instance of WebSocket plus initiating a connection. Only after initializing/opening a connection can you treat such an instance of WebSocket as ready to use from your ClientPool.

Now, you have a problem: you don't want to block your test execution while waiting to initialize a new WebSocketClient. Because of this, this "resize" should be quite smart and invoked in the background before the real need to get WebScoketClient from ClientPool. Basically, we should support some configuration for the Client Pool to start a "heavy" resize process ahead of time.

I like the idea of a resizable Client Pool; we just need to define a simple API for it.

[RomanPoprava]

As an option to implement this - create a buffer of clients for the client pool. Let say we initialized 50 client. If the number of clients in the pool drops below the threshold (e.g. 10% of initial client pool capacity - 5 clients), then don't wait until all clients are used up, but instead, start increasing the pool by adding new 10% of clients. Add maybe is good option will be to do this 'clients adding' in a separate thread to no block the whole execution.

[AntyaDev]

Sounds reasonable to me. Do you think you can implement it?

[AntyaDev]

ping @RomanPoprava

[RomanPoprava]

sorry about delay, missed that message I don't have any background with the F# before. :(

[AntyaDev]

@RomanPoprava, we can have it in C# as a Nuget package if you wish