aviggiano / redis-roaring

Roaring Bitmaps for Redis
MIT License
348 stars 56 forks source link

Create examples and instructions on how to use this module #41

Closed aviggiano closed 5 years ago

aviggiano commented 7 years ago

Since redis modules are relatively new, it would be interesting to add instructions about its usage (also some CLI examples)

FunkyYang commented 6 years ago

nothing new :(

aviggiano commented 6 years ago

Hi @FunkyYang, thank you for checking this module out. Have you tried installing it? Did you find any blockers?

FunkyYang commented 6 years ago

yes.I am going to use this module for compression bitmaps which store in redis, I will report result soon later :)

aviggiano commented 6 years ago

Great, please let me know what you think. If you have any suggestions feel free to submit a PR.

FunkyYang commented 6 years ago

I used bitmaps to store many dimension keys which used for user uids. for example keys format like that:

_ and it's value is user's uids bitmap .and I have to store all of this bitmaps in redis. and not compression by default.
FunkyYang commented 6 years ago

sorry,the key format:dimension_date.and Can you give me some suggestions for this?a little redis memory used and a high query speed ? all of data will not be deleted

aviggiano commented 6 years ago

Hey @FunkyYang

Could you please explain in a bit more detail what your key and values look like? So that I can help you more easily. For example, were you already using Redis native bitmaps before trying this module? How?

A common use case for Redis bitmaps is when you have a list of user IDs and a yes/no state, such as valid/invalid, purchased/not-purchased, accepted/declined, etc. For example, if you have a list of user IDs and each user is mapping to a registered/not-registered value, then you could use redis-roaring to store this as

// the name of my key is registered_users
// user 100 is registered
> R.SETBIT registered_users 100 1
// user 200 is registerd
> R.SETBIT registered_users 200 1

Another great benefit of this library is that bitmap operations is very fast. So if you want, for example, the list of users who are registered AND who accepted the terms and conditions, we could do something like

// the name of my key is accepted_tc
// user 100 has not accepted
> R.SETBIT accepted_tc 100 0
// user 200 has accepted
> R.SETBIT accepted_tc 200 1
// now I need registered_users AND accepted_tc -- this will be stored on the key dest
> R.BITOP AND dest registered_users accepted_tc
// if I query dest, the user 100 will be false and the user 200 will be true
> R.GETBIT 100 
> R.GETBIT 200

I hope this makes things clear. If not, please tell me more about what you are doing with bitmaps and I'll see if I can help.

FunkyYang commented 6 years ago

yes.I used redis native bitmap before use this module .but I need store all kind of uids which store user's state data.the redis memory usage is growing.this module can compres redis native bitmap in a certain extently. I want to use RoaringBitmap[https://github.com/RoaringBitmap/RoaringBitmap] to store my data rather than redis.such as MMAP.lazying load

aviggiano commented 6 years ago

@FunkyYang that's interesting. What kinds of data are you storing? You could have one roaring-bitmap per user per data. Could you share your table schema, so that we can design something more specific?

FunkyYang commented 6 years ago

like this

key1: location_china
value:bitmap[uids]
key2: location_america
value:bitmap[uids]
...

a lot of bitmaps like that abvoe. another sence use redis bitmap to compute user's retention by day.

aviggiano commented 6 years ago

Interesting. Please let me know if you need any help

FunkyYang commented 6 years ago

the redis memory usage with day's growing and no data to be deleted or expired in redis.thus redis memory is run out one day .

aviggiano commented 6 years ago

You think there's a memory leak in this module? Interesting. I'd have to debug it more closely. Can you explain how was your usage? How many keys were used? Do you have a sample dataset we could use?

Thanks

FunkyYang commented 6 years ago

sorry about that.there is no problem with this module .just the sence on work

shivanivaidya commented 5 years ago

I am not able to use it on Docker. I keep getting the error- "error while loading shared libraries:libredis-roaring.so. cannot open shared object file: no such file or directory". Not able to fix it.

aviggiano commented 5 years ago

Hello @shivanivaidya Sorry but I haven't looked at this issue. It seems the docker image is broken, but if you compile the code locally (in the meantime) it should work fine

shivanivaidya commented 5 years ago

Hi! Yes. I am able to run it locally. But I am trying to create a docker image with Redis along with all the modules that I require and I'm not able to do that. It will be nice if you'll be able to fix it sometime soon. Thank you! :)

aviggiano commented 5 years ago

Thanks for the feedback. I'll take a look at this issue by the end of this week.