Support for BF.LOAD command to handle AOF Rewrite

valkey-io / valkey-bloom

Rust based Valkey Module which provides a BloomFilter data type / APIs

9 stars 8 forks source link

Support for BF.LOAD command to handle AOF Rewrite #8

Open KarthikSubbarao opened 2 weeks ago

KarthikSubbarao commented 2 weeks ago

Modules can support AOF Rewrite through a Module data type callback. valkey-bloom implements a module data type callback and during AOF Rewrite, we would want to write a "BF.LOAD" command that contains the dump of the existing bloom object with all the properties and bit vector dump so that it can be recreated when the command is used later on from the AOF File.

Syntax: BF.LOAD <key-name> <dump>

This issue will implement the data type callback here (https://github.com/valkey-io/valkey-bloom/blob/unstable/src/bloom/data_type.rs#L22) and add support for a new BF.LOAD command

wuranxx commented 1 week ago

@KarthikSubbarao I'd like to contribute this feature. Is this already under development?

Is BF.LOAD the equivalent of redisbloom's BF.LOADCHUNK? Do we need to implement BF.LOAD and BF.DUMP (BF.DUMP replaces BF.SCANDUMP) at the same time? Do the two commands need to be consistent with the command names of redisbloom to ensure API compatibility?

KarthikSubbarao commented 1 week ago

@wuranxx - Thank you. This feature is not yet under development, and you can contribute it.

BF.LOAD is intended to provide similar functionality (to restore a bloom object), but it is different from BF.LOADCHUNK.

We only plan to use this during AOF Rewrite. You can take a look at the "AOF Rewrite handling" section in the ValkeyBloom RFC here
It does not load in iterations. It loads / restores a bloom object in one go.

For save (equivalent to BF.SCANDUMP), valkey-bloom provides RDB Save functionality which can be used. Alternatively, when using AOF, we will write the BF.LOAD command as mentioned above