tidwall / summitdb

In-memory NoSQL database with ACID transactions, Raft consensus, and Redis API
Other
1.41k stars 78 forks source link

Q: transactions or pipelined commands? #15

Open glycerine opened 7 years ago

glycerine commented 7 years ago

Does MULTI actually do a transaction, or does only do pipelining? It appears to only save up and then submit a set of commands at once.

In a multi-statement transaction, I expect to be able to read and then write based on that read. In a simple example with two variables x and y representing bank accounts, consistently transfer a balance x -= 10 and y += 10, where no reader in the middle sees an in-progress value. Typically the read is based on locks or an MVCC implementation.

thought: Maybe this is just phrasing thing; the MULTI/EXEC/DISCARD could be described as "PIPELINING" or "PIPELINING A SEQUENCE OF COMMANDS" instead of "TRANSACTION" commands.

Alternatively, are there actual multi-statement transactions available, say with the javascript? I just couldn't figure out how to read the value of a GET after I started a MULTI.

tidwall commented 7 years ago

Does MULTI actually do a transaction, or does only do pipelining?

A pipeline in Summit is limited to the number of commands that can fit into one tcp packet, or whatever network boundary Redcon defines. A client can attempt to send a lot of commands in a single packet, but that packet might end up getting split up into smaller groups.

MULTI on the other hand is a real transaction. It works similar to pipelining but all commands between MULTI and EXEC/DISCARD are guaranteed to be processed as a single group.

Though It's not really useful in the case that you've describe because Summit doesn't support the WATCH command at the moment. Otherwise it's designed to work like Redis transactions. It queues up the commands and commits them all at once. Any command between MULIT and EXEC return +QUEUED or -ERR messages.

Alternatively, are there actual multi-statement transactions available, say with the javascript?

Yes, the javascript EVAL command does handle transactions in the way that you describe. It can read, write, and rollback.

It's designed to work like the Redis EVAL/EVALSHA commands (except JS instead of Lua).

glycerine commented 7 years ago

Wow. I have to say, Summitdb is pretty sweet. I tried the EVAL functionality and ran two conflicting transactions. txn1 was started first, and expected to take about 5 seconds. txn2 was started immediately.

txn1:

127.0.0.1:7481> set zoo 0
OK
127.0.0.1:7481> eval "var x=parseInt(sdb.call('get',KEYS[0]),10); for (i=0;i< 200000;i++) {sdb.call('set',KEYS[0],1+i+x)}; return sdb.call('get',KEYS[0]);" 1 zoo
"200000"
(5.54s)
127.0.0.1:7481> 

txn2:

127.0.0.1:7481> eval "var x=parseInt(sdb.call('get',KEYS[0]),10); for (i=0;i< 200000;i++) {sdb.call('set',KEYS[0],1+i+x)}; return sdb.call('get',KEYS[0]);" 1 zoo
"400000"
(9.57s)
127.0.0.1:7481> 

Excellent! It looks like txn1 successfully obtained the write lock, and txn2 simply paused until txn1 released it. Nice.

I notice that errors in script abort the transaction. Very nice.

127.0.0.1:7481> get zoo
"600000"
127.0.0.1:7481> eval "var x=parseInt(sdb.call('get',KEYS[0]),10); for (i=0;i< 100000;i++) {sdb.call('set',KEYS[0],1+i+x)}; return sdb.call('rollback');" 1 zoo
(error) ERR unknown command 'rollback'
(2.94s)
127.0.0.1:7481> eval "var x=parseInt(sdb.call('get',KEYS[0]),10); for (i=0;i< 99999;i++) {sdb.call('set',KEYS[0],1+i+x)}; return sdb.call('discard');" 1 zoo
(error) ERR command not allowed from script 'discard'
(2.48s)
127.0.0.1:7481> get zoo                         # unchanged after both the above!
"600000"
127.0.0.1:7481> 

Is there a recommended way to abort a transaction and not return an error? I'm not sure when that would be useful. But the error seems vaguely out of place when my code intended to abort. Are there examples of using eval someplace I could study for guidance?

tidwall commented 7 years ago

Wow. I have to say, Summitdb is pretty sweet.

:) Thanks for the kind words.

Is there a recommended way to abort a transaction and not return an error?

There's currently no way to rollback without throwing an error. Right now the best way is to throw a javascript error with a specified value and then read that error back with the client.

127.0.0.1:7481> eval "sdb.call('set',KEYS[0],10);throw 'my rollback message';" 1 zoo
(error) my rollback message

Perhaps it would be more ideal if there was a sdb.rollback() function?

Are there examples of using eval someplace I could study for guidance?

The only examples online that I know of are on the EVAL Wiki page. It's designed to be very simple. There's really only two added javascript functions, sdb.call() and sdb.pcall(). You can call practically any Summit command from one of the those two functions (with the exception of RAFT*/MULTI/EXEC).

I recommend that you keep the javascript as procedural as possible, avoid JS classes. And use SCRIPT LOAD and EVALSHA to cache often used scripts.