EchoVault / SugarDB

Embeddable and distributed in-memory alternative to Redis.
https://sugardb.io
Apache License 2.0
425 stars 25 forks source link

Implement SCAN command #108

Open kelvinmwinuka opened 2 months ago

kelvinmwinuka commented 2 months ago

Scans the keys in the currently selected database. Reference: https://redis.io/docs/latest/commands/scan/

Client-Server Spec:

Command File: ./internal/modules/generic/commands.go Test File: ./internal/modules/generic/commands_test.go

Command: scan Module: constants.GenericModule Categories: contants.KeyspaceCategory, constants.ReadCategory, constants.SlowCategory Description: (SCAN cursor [MATCH pattern] [COUNT count] [TYPE type]) scan the keys in the currently selected database Sync: false

Embedded Spec:

Command File: ./echovault/api_generic.go Test File: ./echovault/api_generic_test.go

Documentation

Add documentation to ./docs/docs/commands/generic/scan.mdx

NicoleStrel commented 1 month ago

Hi @kelvinmwinuka! I'd like to work on this if it's free

kelvinmwinuka commented 1 month ago

Hey @NicoleStrel this is free, you can pick it up.

NicoleStrel commented 1 month ago

@kelvinmwinuka Hey! So I started looking into this, and for the SCAN command,it iterates using a cursor index and returns another cursor index to continue iterating later on.

So, we need to keep track of the order of the kvstore so that on the next call the user can retrieve the next keys

I was thinking of adding an index attribute in KeyData - likely that would change a lot of the code though.

type KeyData struct {
        Index int 
    Value    interface{}
    ExpireAt time.Time
}

thoughts/any other ideas?

kelvinmwinuka commented 1 month ago

I don't mind this solution. As long as we make sure to avoid index clashes. I don't think this will break anything.

Try it out and we'll examine it in a code review.

NicoleStrel commented 1 month ago

or actually, I saw the discussion about the skip list - https://github.com/EchoVault/SugarDB/discussions/139

since it also maintains order (O(N)) this could work in place of the map of the KeyData struct - what do you think?

adding the index/potentially using the skip list should probably be done first before the SCAN implementation

guycipher commented 1 month ago

@NicoleStrel a skip list probably wont surpass the speed of a go map. For a data structure that keeps things sorted yeah a skip list could work well but in my tests go's map is blazing fast if key's don't have to be sorted, it depends how you'd want to do it. I'm not too familiar with Redis to be completely honest with y`all. @kelvinmwinuka knows this :P

guycipher commented 1 month ago

If you want a really fast in-memory sorted data structured a BStar tree or BStarPlus tree would be amazing. I am working on this on the side and the write and read speed surpass that of a skiplist in-memory. Once I get confident in the implementation I will share, this could work in SugarDB as an optional data structure, possibly? Skiplists are easier, surely :P