Redis Timeout

[Kmaczek]

Here is the message: Can't execute command: Timeout performing GET (60000ms), inst: 0, qu: 1, qs: 0, aw: False, bw: SpinningDown, rs: NotStarted, ws: Idle, in: 0, serverEndpoint: xxx.redis.cache.windows.net:6380, mc: 1/1/0, mgr: 10 of 10 available, clientName: XXX(SE.Redis-v2.6.70.49541), IOCP: (Busy=0,Free=1000,Min=12,Max=1000), WORKER: (Busy=0,Free=32767,Min=12,Max=32767), POOL: (Threads=7,QueuedItems=0,CompletedItems=201), v: 2.6.70.49541

I'm wondering what might be the issue. I'm are using Azure Redis, the same code was working fine few months ago. I have this timeout ,synctimeout=60000 And I'm running just one get command.

[NickCraver]

If you upgrade to latest, you'll likely see 2 things:

1. The connection never connected
2. Why it never connected :)

This is likely a configuration issue, either in the Redis connection or firewall or some such.

[Kmaczek]

@NickCraver Yes, that was firewall setting, thank you. This is only a side issue for me, as I used that connection to test/debug main issue, unsuccessfully though, only until recently I was able to figure what it is about.

Issue is as follows: One of the entries, lets say with key products, has increased in size a lot (10MB now). It takes Redis 8-10 sec to return them (a lot of this time is probably because servers are far). Main app is requesting independent endpoint which ask for products 6 times, and also doing a lot of other smaller requests that usually take around 100ms. All of above requests are triggered when user opens homepage which needs to load a lot of stuff to initialize. 60 sec timeout is not enough here. Requests timeout seem to be adding to previous requests. E.g.

First GET products will take 8 sec Second products will take 16 sec Third products 24 sec and so on

With all the small requests it adds to 60 sec sometimes, resulting in timeout. I'm wondering if this is because of pipelining and if this can be switched somehow so timeouts are counted independently of each other, or maybe we are doing something wrong here?

While writing this I found Redis Timeouts and I will try to follow Too many requests section, but any input is welcomed.

[NickCraver]

@Kmaczek I don't think it's your number of requests such much as the size of requests, 10MB is pretty huge to continually take back and forth. When in Redis we're typically dealing with sub-millisecond for operations, that's a large amount.

Consider a 1Gb/s connection - we can't really think about that in seconds, but rather in 1000Mb/s or 1Mb/ms. In those terms, you're transferring 10*8 Mb which means an 80ms occupation of the connection assuming max speed and no other latency, or at most 12.5 transfers per second of that key.

You can MONITOR on the server to see what's happening, but I'd be looking at your bandwidth meters, I bet you're maxing out the link.

[Kmaczek]

@NickCraver I think you are right, we use C0 Azure Plan for this environment It has only 5Mbps

one product = 10MB / 5Mbps = 16 seconds

We pull that products 6 times and timeout happens.

I think that solves it, thanks for your help @NickCraver