rosedblabs / rosedb

Lightweight, fast and reliable key/value storage engine based on Bitcask.
https://rosedblabs.github.io
Apache License 2.0
4.58k stars 632 forks source link

Proposal: kv db base on remote file system #313

Closed izouxv closed 3 months ago

izouxv commented 4 months ago

why i want to do this

kv db base on remote file system. many rosedb instant can r&w the db file without service. special for mobile app without backend service.

how do i do it

use afero.Fs to replace os.Fs. it can support local or remote fs, and any other type fs use Lock interface to replace local file lock. i can support local or remote lock use db interface to replace struct split the read write api of db api. write with the lock, read without lock

type DBReadOnly interface {
    Close() error
    Fs() afero.Fs
    Stat() *Stat
    Get(key []byte) ([]byte, error)
    Exist(key []byte) (bool, error)
    Ascend(handleFn func(k []byte, v []byte) (bool, error))
    AscendRange(startKey, endKey []byte, handleFn func(k []byte, v []byte) (bool, error))
    AscendGreaterOrEqual(key []byte, handleFn func(k []byte, v []byte) (bool, error))
    AscendKeys(pattern []byte, filterExpired bool, handleFn func(k []byte) (bool, error))
    Descend(handleFn func(k []byte, v []byte) (bool, error))
    DescendRange(startKey, endKey []byte, handleFn func(k []byte, v []byte) (bool, error))
    DescendLessOrEqual(key []byte, handleFn func(k []byte, v []byte) (bool, error))
    DescendKeys(pattern []byte, filterExpired bool, handleFn func(k []byte) (bool, error))
}
type DB interface {
    DBReadOnly
    Sync() error
    Watch() (<-chan *Event, error)
    Put(key []byte, value []byte) error
    PutWithTTL(key []byte, value []byte, ttl time.Duration) error
    Delete(key []byte) error
    Expire(key []byte, ttl time.Duration) error
    TTL(key []byte) (time.Duration, error)
    Persist(key []byte) error
    DeleteExpiredKeys(timeout time.Duration) error

    NewBatch(options BatchOptions) *Batch

    Merge(reopenAfterDone bool) error
}

type Lock interface {
    TryLock() (bool, error)
    Unlock() error
}
roseduan commented 3 months ago

Thanks for your proposal.

Are you going to use this new feature in your project?

izouxv commented 3 months ago

yes

my project is developing it is a app that without online server only the remote file system. this feature can support it

roseduan commented 3 months ago

yes

my project is developing it is a app that without online server only the remote file system. this feature can support it

Thanks, but I think this is not a common feature for most users. But you can maintain this yourself. I don`t have enough motivation to add it to main branch.

izouxv commented 3 months ago

ok