search

twitter / twemproxy

A fast, light-weight proxy for memcached and redis
Apache License 2.0
12.15k stars 2.06k forks source link

May I use redis-bloom modules with twemproxy? #657

Open GuanFoxyier opened 3 years ago

GuanFoxyier commented 3 years ago

twemproxy has been used by us.

Now we want to use the bloom filter modules.

What should I do?

TysonAndre commented 3 years ago

Currently, no, but wrapping with https://redis.io/commands/eval can be used to invoke those commands.

The issue would be that twemproxy isn't aware of the signature of the commands, so it doesn't know if a command has multiple keys, needs to be fragmented to multiple servers, is synchronous and able to be forwarded (i.e. not pubsub), etc. https://oss.redis.com/redisbloom/Bloom_Commands/ and related pages would be useful as a reference for anyone implementing that, this seems to be single-key commands

631086083 commented 2 years ago

hello， @TysonAndre @GuanFoxyier
I implemented a function that supports the bloomfilter module in https://github.com/631086083/twemproxy/tree/bf, and wrote some test cases, which are running smoothly on some machines.

This kind of requirement increases the number of string matching for not needing this module, which may have a little impact on performance, whether it needs to be merged into the main branch or as an independent branch.

For keys, the commands in the https://oss.redis.com/redisbloom/Bloom_Commands/ document are all single-key, so the proxy can focus on forwarding and does not need to be split into multiple shards.