apache / opendal

Apache OpenDAL: One Layer, All Storage.
https://opendal.apache.org
Apache License 2.0
3.45k stars 484 forks source link

`scan` for redis service #3500

Open TD-Sky opened 1 year ago

TD-Sky commented 1 year ago

Wanted feature

The Operator built from Redis could use scan to list entries matching the glob-style pattern of redis.

Real scene

There is a function called move registration in our system. Users register via the function where the uploading file should be move to after its transmission done. Every registration record has its own ID (move_id), and every file also has its own ID (meta_id). Then we format them into movreg_{move_id}_{meta_id} as the key in redis.

We may have only one of the two IDs to query the value. For example, we have only meta_id 832917 and it was stored as movreg_432432_832917; to get the value, we need to excute:

> keys movreg_*_832917
"movreg_432432_832917"

> get movreg_432432_832917

That's why we need glob-style scan in opendal; we want to get all the value whose key matches the given pattern.

Conjectures about implementation

I think KEYS or SCAN can work. Both of them could do glob-match on whole database. But KEYS is slower in general.

Xuanwo commented 1 year ago

This feature request is related to https://github.com/apache/incubator-opendal/issues/1251.

Maybe it's possible for us to add glob and regex support for list, so users can use opendal in this way:

let entries = op.list_with("prefix/").glob("ab[1-9][a-b]").await?;

This is a zero-cost abstraction because: