New Condition SortedSetNotStartsWith

[ArnoKoll]

I am using sorted-sets as indices to hashes. For me, it would be useful to have a new condition:

SortedSetNotStartsWith (Key, Member)

where Member is a string and only a part of an existing Member.

A transaction should be discarded, if in a SortedSet a member exists, that starts with the member-parameter. I hope, this is only a small extension to the condition-class.

The background is: Members of my sorted-sets could for example be composed of the value of a hashfield „custno“ and the value of a hashfield „id“ which is also part of the hash-key, e.g.: 12345:8015 (= custno 12345, id 8015) 14526:19520 (= custno 14526, id 19520)

Given a sorted-set-key „CUST-NO“, two concurring processes could possibly try to insert a new customer with the same customer-no (e.g. 22055). Each process generates a unique id before then start of the transaction (e.g. 19521 and 19522), so the following members would be added to the sorted-set: Process1 : 22055:19521 Process2: 22055:19522

The value of hashfield „custno“ should be unique, but if process1 first exits his transaction, in process2 a condition SortedSetNotContains("CUST-NO", "22555:19522") would not discard his transaction (and vice versa).

Therefore a new condition SortedSetNotStartsWith("CUST-NO“,“22555:“) would be perfect. Of course, the name of the condition is only a suggestion.

[mgravell]

We need to limit ourselves to operations that exist in redis; "starts with" - not a thing for sorted sets. How would you do this in pure redis, perhaps in redis-cli? If you can do that, the next step is probably "use Lua - port your redis-cli to a Lua script"

[mgravell]

As an aside: redis has fairly limited operations. Often, you can't simply say "this is how my data is structured, and this is what I want to do: what's the operation to do that?"- instead you need to ask "this is what I want to do, and here's the operations that exist; how do I structure my data?"

[ArnoKoll]

Hello Marc,

ok, so i took a dive into the code.

First my respect, i see, you have to deal with many complex themes and issues, so i understand you have to limit your resources.

In the end, it wasn’t very complicated to realize the wanted extension. Adding a new condition, executing a ZRANGEBYLEX command with „LIMIT 1“, did the job, then the result is compared with the search-value. To avoid any encoding-problems within stackexchange.redis, the parameter for the search-value is a byte-array and any encodings must been done in the calling application. The system load of this call should be rather small by the limitation of the result count.

I'll send you a pull request, so you can decide about adding the extension to the standard lib.

To your second comment: I am using redis in a production environment without any problems for some years, simulating tables and indices, and it doesn’t feel as if overwhelming or misusing redis. Of course i check for duplicate entries (in my context) during the compilation of a transaction and in my practice this was good enough. But now the descibed extension also gives me the theoretical security that in a heavy environment no insertion of an identical entry (in my context) by a concurring process is unnoticed in the timespan between duplicate-check and execution of the transaction.

[mgravell]

Interesting approach! For what it is worth, personally I'd probably still advocate for Lua here, just: Lua that executed the ZRANGEBYLEX; Lua is almost always easier to get right than redis MULTI/EXEC, plus it doesn't stall concurrency on the multiplexer (which SE.Redis needs to do here, especially when using constraints).

[ArnoKoll]

Ok, i understand your objection. I’ll try the Lua-solution later, to avoid integrating the new condition every time, when i want to use updates of your lib. Until then, i’ll give my individual version a chance, because i fixed a bug in the zrevlexbyrange-command here (see my first pull-request) and the new condition for me is easy to use.