search

Netflix / dynomite

A generic dynamo implementation for different k-v storage engines
Apache License 2.0
4.2k stars 531 forks source link

Select "index" breaks connection on Redis #492

Open metal3d opened 7 years ago

metal3d commented 7 years ago

Hi, We created https://github.com/Smile-SA/dagota that makes easy to setup a Statefulset on Kubernetes and OpenShift. It works as expected.

But as soon as we tried to use it with some clients (PHP frameworks, Go, Bash) a problem appears when some modules, extension or addons want to use several databases using "select " where is an integer.

We're not sure about why Dynomite cannot use that command, but is there any chance that could work ?

Without that, it seems to be difficult to use it with standard frameworks that want to split key/val in several databases to be able to flush them selectively.

Unfortunatly, we cannot "fix" frameworks to use only one db (and so, remove the entire "select" commands).

Anyway, congratulation for your work !

shailesh33 commented 7 years ago

Yes we do not have multiple DB support. But we are open to ideas and contributions.

metal3d commented 7 years ago

Ok, so it's not a problem from our project :)

I didn't checked how you proxify redis command to the redis server, but I wonder why it is not possible. AFAIK this is only a command to send to redis server to say "hey, I'm writing to that db" for a certain transaction.

I probably need to read source code of Dynomite and try to integrate that, but it's not easy...

Any particular part of source code to check that you can point me ?

shailesh33 commented 7 years ago

The problem would be that we have to make sure that we correctly, precisely and guaranteed-ly change the DB context of the underlying Redis datastore. Other than that, it is not really a big change. One has to consider which is the current DB selected and if it is different, insert a "Select <>" statement in the queue of messages to the Redis server (and of course peers). This is not just local Redis server, but also across replication to other nodes. Making sure that this request never gets dropped.

metal3d commented 7 years ago

Ok, I took a look on source code and I now understand what does Dynomite. What you say is clear, thanks for your answer.

So, as now I undernstand, Dynomite doesn't only make a "proxy" but you treat requests.

Sadly, I'm not good enough in C to help or propose PR. But AFAIK there are so many clients that need to select different db (and that make "select 0" by default, that breaks dynomite connection) so I think that could be a very usefull enhancement.

One more time, I'm impressed by Dynomite. Our Openshift/Kubernetes project works as expected but we can't use it for a lot of projects... That's sad :)

arocki7 commented 5 years ago

We are having similar issues. Tried to use Dynomite along with 3Scale and Select Index failed.