search

Netflix / dynomite

A generic dynamo implementation for different k-v storage engines
Apache License 2.0
4.19k stars 530 forks source link

Dynomite and Redis streams #668

Open jkimalane opened 5 years ago

jkimalane commented 5 years ago

I'm wondering, are there any major show stoppers why streams commands can't be handled by dynomite? Maybe someone who is more deeply involved with dynomite and has deep understanding of dynmoite internals can elaborate this bit more. If it's not that hard and does not require massive amount of changes, I could try to implement those X for adding support for streams data type.

jkimalane commented 5 years ago

I'm bit stuck as I can't figure out how multibulk responses are forwarded from redis server through dynomite proxy to redis client. Commands like XINFO STREAM or XRANGE return multibulks and according to comments in dyn_redis.c only commands SSCAN/HSCAN/ZSCAN are handled. To me it seems that recursion is needed to handle this case.

jkimalane commented 5 years ago

There is a pending pull-request #675 which provides basic streams support. Although there is a timing issue but in general, all streams related commands are supported.

ipapapa commented 5 years ago

@smukil was working on some changes in this area. I will let him handle this issue.

jkimalane commented 5 years ago

any news about this one?

smukil commented 5 years ago

@jkimalane We're strapped for bandwidth at the moment unfortunately, but we're in the process of on-boarding more developers on Dynomite, so we should be able to move faster in the coming weeks and get the patch in.