Open raphaelauv opened 2 hours ago
@raphaelauv Hi, We discussed it when creating the cluster scan, which is not usual scan. Can you explain your use-case? You happened to have a wrong state of the cluster and wanted to keep scanning or is it a state you aware of ahead?
Less relevant, but the improvement of scan with match can be done (internally by Valkey) per node, but not per cluster. At this point there's no built in functionality of wide scanning of a cluster.
hi, thanks @avifenesh
we want our scan to work even with missing nodes (we have big cluster(s) and a missing node is a common thing)
would be great that we can explicitly use scan with a cluster-require-full-coverage
at false
we want our scan to work even with missing nodes (we have big cluster(s) and a missing node is a common thing)
The challenge with missing nodes and cluster scan is that in such cases we can't provide any guarantee about the validity of the scan.
Since currently there's no cluster wide scan provided by Valkey, we need to create some complex logic to provide this kind of scan with the guarantees of scan. The way to do that is to track the covered slots. In case a cluster has missing slots, there's no way we can validate that the scan is over and everything is covered.
The solution in such a case would probably be to iterate blindly over the cluster nodes and to ignore nodes that have connection issues, without giving guarantees, and users will need to choose this scan type ahead with an optional flag or so.
In your case, you would prefer to use a scan with no guarantees over a scan with guarantees which can't scan a not covered cluster?
I'm just trying to get the picture and to understand the needs better.
If you are fine with sharing this information, i would also like to hear why a case of missing nodes (a full shard, in case you have replicas) is common. I have some experience, and missing shards are not supposed to be common. If you prefer to take the specific privately, we can chat in Valkey discord, and you can approach me there in private as well.
The solution in such a case would probably be to iterate blindly over the cluster nodes and to ignore nodes that have connection issues, without giving guarantees, and users will need to choose this scan type ahead with an optional flag or so.
yes this is the missing feature
Describe the bug
When I scan a redis cluster with a missing node , the scan do not work even if the redis cluster is setup with
cluster-require-full-coverage no
Expected Behavior
with
cluster-require-full-coverage no
a redis scan should work with a missing node , like this ->Current Behavior
scan fail with a log on the missing node
Reproduction Steps
Possible Solution
No response
Additional Information/Context
No response
Client version used
1.1.0
Engine type and version
8.0.1
OS
ubuntu 22.04
Language
Python
Language Version
3.11
Cluster information
No response
Logs
No response
Other information
No response