Scan not return all keys on Redis Cluster

[frgao9]

Hi ioredis team, I'm using ioredis to scan and delete keys that matched with some prefix in Redis Cluster (AWS MemoryDb), see sample code below

let cursor = '0';
do {
        const res = await redisClusterClient.scan(cursor, 'MATCH', `{prefix}:*`, 'COUNT', 100);
        // redisClusterClient.unlink(...res[1]);
        cursor = res[0];
} while (cursor !== '0');

However i observed that scan not always return all matched keys though cursor eventually is '0'. While this works correctly if I create redis client by just connecting to a single Redis node.

Appreciate your time and effort in advance!

[leibale]

using ioredis cluster client:

• if the command has keys in the arguments - the command will execute on the node that "owns" the keys slot.
• otherwise - the command will be executed on a random node.

in the case of SCAN, the command arguments do not contain any information about the key - so it will be executed on a random node. In order to scan the whole cluster you'll need to run the script above for all shards (either masters or replicas).

// one by one
for (const client of redisClusterClient.nodes('master')) {
  await scanNode(client);
}

// in parallel
await Promise.all(
  redisClusterClient.nodes('master').map(client => scanNode(client))
);

function scanNode(client) {
  let cursor = '0';
  do {
    const res = await client.scan(cursor, 'MATCH', `{prefix}:*`, 'COUNT', 100);
    // client.unlink(...res[1]);
    cursor = res[0];
  } while (cursor !== '0');
}

(for the sake of simplicity I've used masters in this example, but you can use replicas as well if the script is read-only. just make sure to not run it on more than 1 replicas from the shard.)

(BTW, you can use FLUSHDB or FLUSHALL to remove all the keys, there is no reason to SCAN and UNLINK)

[frgao9]

thank you @leibale!