Exceptions when running load test

[bapti]

Hi there,

I was wondering if I could get a little help with some exceptions that crop up when I run a load test against my site.

During the test I see a lot of RedisClient is not connected.

System.InvalidOperationException: RedisClient is not connected
   at ctstone.Redis.RedisClient.Write[T](RedisCommand`1 command)

An example of the code that causes this

    public Member Get(int userId)
    {
        using (var client = redisClientFactory.Create())
        {
            var result = client.Get(key(userId));
            return String.IsNullOrEmpty(result)
                ? null
                : JsonConvert.DeserializeObject<Member>(result);
        }
    }

    public RedisClient Create()
    {
        return new RedisClient(host, port, timeoutInMilliseconds);
    }

From what I can tell this is fairly standard usage no?

Thanks Neil

[bapti]

I've also found this exception

System.Net.Sockets.SocketException
Only one usage of each socket address (protocol/network address/port) is normally permitted 192.168.35.167:6379

System.Net.Sockets.SocketException (0x80004005): Only one usage of each socket address (protocol/network address/port) is normally permitted 192.168.35.167:6379
   at System.Net.Sockets.Socket.DoMultipleAddressConnectCallback(Object result, MultipleAddressConnectAsyncResult context)
   at System.Net.Sockets.Socket.BeginConnect(String host, Int32 port, AsyncCallback requestCallback, Object state)
   at ctstone.Redis.RedisConnection.Connect(Int32 millisecondsTimeout, Int32 readTimeout)
   at ctstone.Redis.RedisClient..ctor(String host, Int32 port, Int32 timeoutMilliseconds)
   at BrightSolid.Lives.Cache.Infrastructure.RedisClientFactory.Create()

Perhaps that might shed some light on what's going wrong

[bapti]

I'm going to write a pooled connection/client factory - would you be interested in taking something like this as a contribution?

[ctstone]

Absolutely! I would be more than happy to merge in a connection pool.

To your original question, how many connections are you opening and in what span of time? I read a few posts about dynamic port exhaustion, possibly related to keep-alive.

Any reason you can't reuse the RedisClient for multiple requests? Instantiating RedisClient many times will most certainly slow you down due to overhead in the constructor/disposal. If you are creating many instances due to multi-threading, perhaps it would make more sense to switch to RedisClientAsync?

[bapti]

In that case perhaps I'm going about my redis clients wrong. I was newing one up for every request. Perhaps I'd be better putting my client to one-per-web-request and injecting it directly rather than injecting the factory and resolving new clients for each redis operation.

I'll keep working on my connection pool manager and once I'm happy with it create a PR

[ctstone]

For web clients, I usually lazy-load a RedisClient instance into HttpContext.Current.Items[key], which is supposed to be a safe place to store singletons on a per-request scope. Here's a sample factory class:

public static class Current
{
    public static RedisClient RedisClient
    {
        get { return Get("Redis", () => new RedisClient(...)); }
    }

    static T Get<T>(string key, Func<T> func)
        where T : class
    {
        return HttpContext.Current.Items[key]
          ?? (HttpContext.Current.Items[key] = func());
    }
}

Then you can just call Current.RedisClient.Whatever() from anywhere.

[bapti]

That's good to know, I use castle windsor so I'll implement something similar and inject the redis client where I need it for that web request instead of resolving a new one for each operation.

I'm also writing the a client manager as I have a cluster of 3 servers with sentinel so I'll try and figure all that out over the next week as writes have to go to the master and reads from the slaves (or so the theory goes).

I have to say I really like your library, your code is all easy to follow and clean, thanks for all your efforts!

[bapti]

I'm getting stuck into the connection pool now. Other than manually setting a client to busy and not busy

    public class PooledRedisClient : RedisClient
    {
        public bool Busy { get; set; }
        public PooledRedisClient(string host, int port, int timeoutInMilliseconds) 
            : base(host, port, timeoutInMilliseconds) { }
    }

Is there a way from within RedisClient or RedisConnection which I could use to see if a connection is in use. I guess I could do it from within castle windsor attaching to the per web request events but I'd rather be able to encapsulate this behavior inside the ClientPool that I'm creating.

Any advice would be much appreciated. Thanks Neil