search

Netflix / dynomite

A generic dynamo implementation for different k-v storage engines
Apache License 2.0
4.2k stars 534 forks source link

Support SCRIPT LOAD and EVALSHA in Dynomite #624

Closed smukil closed 5 years ago

smukil commented 5 years ago

Dynomite has the ability to deal with scripts that take a single key through EVAL. Users of Dynomite have requested the option of using EVALSHA so as to not resend the script every time. This is made possible by this patch which supports the saving of scripts through SCRIPT LOAD and the execution of them through EVALSHA.

Adding support for SCRIPT LOAD poses a challenge because there is no node(s) that have the right to store a script since the script doesn't evaluate to a token in the token range managed by Dynomite. To deal with this, we need to save the script to EVERY node in the cluster. A new routing mechanism is introduced in Dynomite which sends a message from one server to ALL other servers in the cluster; i.e. it sends to to each node in each rack in each DC. In the future this mechanism will be modified to have a 2 hop approach so as to avoid sending data to multiple nodes across DCs.

Parsing "SCRIPT LOAD" also posed a challenge since the "SCRIPT <LOAD/KILL/FLUSH/EXISTS>" commands are the only commands with a space as part of the command. The parser has been modified to deal with this by tricking the parser to process each keyword as though it is a new command, and then figuring out after parsing the second word, which command it is.

A test case has been added to test both SCRIPT LOAD and EVALSHA.

ipapapa commented 5 years ago

A few more things that need to be done is (a) to update the notes for the supported Redis commands and (b) probably add some documentation about the behavior in the wiki.

robzienert commented 5 years ago

👀 I love that this is here, but I'm not qualified to review. :)

smukil commented 5 years ago

A few more things that need to be done is (a) to update the notes for the supported Redis commands and (b) probably add some documentation about the behavior in the wiki.

Good points. I updated the Redis commands page and updated the wiki as well.