How to restore a Redis Bloom Filter key from an RDB file to a remote redis server？

[Damanchen]

We can use ‘set’ to restore a string type key from RDB file to remote redis server, or use ‘hset’ to restore hash type key ...

But, how to restore a bloom filter key from RDB file to remote redis server ?

According to https://github.com/sripathikrishnan/redis-rdb-tools and RedisBloom source code, I can get the following information about bloom key in rdb file:

RDB binary file about bloom filter key: 0201 0201 0205 0202 0240 6404 7b14 ae47 e17a 743f 0208 04e9 862f b235 0e26 4002 4480 0200 05c3 3340 9001 0000 e024 0000 01e0 002d 0002 e000 09e0 0700 0004 e000 1000 08e0 0009 0010 e000 0900 20e0 0009 0040 e000 0900 80c0 0901 0000 0201 00

means: size 0201 nfilters 0201 options 0205 growth 0202 entries(capacity) 0240 64 error(ratio) 04 7b14 ae47 e17a 743f hashes 0208 bpe 04e9 862f b235 0e26 40 bits 02 4480 n2 0200 Chunk ? 05c3 3340 9001 0000 e024 0000 01e0 002d 0002 e000 09e0 0700 0004 e000 1000 08e0 0009 0010 e000 0900 20e0 0009 0040 e000 0900 80c0 0901 0000 size 0201 end 00

After lzf_decompress of the part of 'Chunk' , we can get following content: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00 \x00\x00\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00

It is as same as the content we get by 'bf.scandump bfkey 1' command，maybe I can restore this part of content to remote redis server by 'bf.loadchunk bfkey xx' ; （Am I right ??）

The question is how to restore other information about bloom filter key in the RDB file, can enyone help me ? thanks a lot ~

[gkorland]

You can use DUMP/RESTORE. https://redis.io/commands/dump https://redis.io/commands/restore

[Damanchen]

Thanks for your promptly reply.

When redis server is running, 'dump' and 'restore' command can do this;

But, if I want to restore data from an RDB file to a remote redis server (Such as a disaster recovery scenario), 'dump' and 'restore' is useless...

[gkorland]

if you already have an RDB just load the server with this RDB

[Damanchen]

I want to restore RDB file to a remote Redis server, not local server, so can't just use RDB file to load.

I would like to know if there is a command to set the RDB content about the Bloom filter key to the remote Redis server. For example, we can use 'set' command to set a string key parsed from an RDB file to a remote Redis server, or use 'hset' command to set a hash key .

[Damanchen]

Do I have to modify the source code to implement this feature? Any better suggestions?

[ashtul]

@Damanchen RedisBlomm has BF.SCANDUMP and BF.LOADCHUNK which you can find here. You can then save the filter, transport it and load it into another server.

[Damanchen]

@ashtul Thanks for your reply.

When the Redis server is running, BF.SCANDUMP and BF.LOADCHUNK can save the filter, transport it and load it into another server.

But, when I only have the rdb file ,how can I load the bloom filter key to a remote Redis server ??

Has anyone encountered similar situations and problems?

[rvencu]

I believe the answer would be: make a local instance from rdb file then scandump locally and loadchunk remotely... I played with a python script that can be easily adapted to loadchunk remotely when the dump is saved on some local files https://github.com/rvencu/crawlingathome-gpu-hcloud/blob/main/helpers/bloomexport.py

[631086083]

hello，I don’t know if it’s late. I wrote a program to load BloomFilter from offline rdb to remote Redis. It is currently under development and testing. @Damanchen https://github.com/KCacheTeam/redis-proxy-resharding