search

thisisdevelopment / helium-route-updater

Automatically adds / removes euis from your helium routes based on updates from your LNS
MIT License
6 stars 2 forks source link

The application does not work with redis cluster #6

Open MertEmirKaya opened 4 months ago

MertEmirKaya commented 4 months ago

Hi, I was using helium-route-updater with a single Redis instance everything was fine so far, but I started to use Redis cluster in my chirpstack application, apparently it does not work with redis-cluster but there is a strange behavior.

This is my LNS_LISTEN value for a single Redis: redis://<my-single-redis-ip-address>:6379/0

after starting to use the Redis cluster I changed the value to: redis://<my-redis-cluster-ip-address>:6379/0

I am getting the following error in the logs: CROSSSLOT Keys in request don't hash to the same slot

When I change the URL to this format(Only added s): rediss://<my-redis-cluster-ip-address>:6379/0 or rediss://<my-redis-cluster-ip-address>:6379

I can confirm that the app can start and function for the first several seconds but crashes eventually. The following is the logs:

Starting
[sync] starting full sync
[sync] Found 3 euis in our helium route
[sync] Found 0 skfs in our helium route
[sync] Found 3 devices in the lns
[sync] euis are in sync
[sync] skfs are in sync
2024/05/15 15:35:25 context deadline exceeded

Since I am using Kubernetes, the pod keeps restarting and crashing again and again.

ederuiter commented 4 months ago

Hi @MertEmirKaya,

The strange behaviour you are describing is the way redis handles clustering. Each redis node in the cluster is assigned a slice of all the keys. If you request a key that is not handled by this node, it will give an error like the one you describe. It is up to the client to handle this and try again on another node (or even better, know upfront and do the request on the right node).

Currently we are using the "standard" redis client from go-redis v9. This client is not cluster aware. There is a universal client that we could use (https://redis.uptrace.dev/guide/universal.html), that has that support .. but that requires some extra configuration.

Currently we have no need for redis cluster support ourselves, but we are happy to take a PR for it