StackExchange / StackExchange.Redis

General purpose redis client
https://stackexchange.github.io/StackExchange.Redis/
Other
5.85k stars 1.5k forks source link

Timeout connecting to bitnami redis cluster hosted on GKE where passing 6 lb dns:30690 #2411

Open rajiv36 opened 1 year ago

rajiv36 commented 1 year ago

We have deployed Bitnami Redis cluster on GKE. After deployment we got 6 external lb services name which then we mapped with dns and are of type dns0:30690, dns1:30690, dns2:30690, dns3:30690, dns4:30690, dns5:30690. 6 pods with 3 master and 3 slave is created with redis cluster deployment by bitnami redis cluster chart with above 6 external ips.

We use stack exchange redis client pkg to read/write on our redis cluster from outside GKE hosted apps.

Do we need to provide all 6 lb dns names in connect method or just one is fine? as when I just pass one dns name it start writing data to redis cluster but on providing all 6 dns names in connect method it throws timeout exception.

Connect method passing with one dns: ConnectionMultiplexer.Connect(6dns names

Connect method failing with timeout: ConnectionMultiplexer.Connect([6dns names](dns0:30690, dns1:30690, dns2:30690, dns3:30690, dns4:30690, dns5:30690) Error: (Increasing timeout doesnot help) ExceptionStack = [StackExchange.Redis.RedisTimeoutException: Timeout performing EXISTS (5000ms), inst: 0, qu: 0, qs: 0, aw: False, bw: Inactive, rs: ReadAsync, ws: Idle, in: 0, last-in: 4, cur-in: 0, sync-ops: 1, async-ops: 2,

Is it right to think that passing just one dns0:30690 internally searches

NickCraver commented 1 year ago

It all depends on how their cluster behaves. If CLUSTER NODES returns all the endpoints to connect to, we'll see that and connect to them. If it doesn't, we won't know about them - some providers do tricky things here or return endpoints that are internal there and not the thing you'd connect through (e.g. the load balancer). Since I'm not familiar with Bitnami or GKE, I can't say for sure, but that's what you're looking at to test.

FYI: there's a second argument to .Connect(Async) that takes a logger - if you pass it a TextWriter you can see a full detail of what's happening on each node - I think that'd be pretty enlightening for you on what's happening in each case here :)