Mutiple atomic transactions as a pipeline

[alishamohanty]

Hi @ptaoussanis ,

I am currently working on a project that needs to send multiple atomic txs as a pipeline. I checked the official documentation but could not found anything helpful. I tried to wrap car/atomic over wcar*

(wcar* conn :as-pipeline (doseq [e elements]
                             (car/atomic
                               conn atomic-lock-attempts
                               (car/multi)
                               (car/lrem list 1 e)
                               (car/rpush list e))))

which workes but the query was sent over multiple network call instead of one.

There was another way to achieve the same using multi and exec with wcar like

(wcar* {} :as-pipeline (doseq [e elements]
                                  (car/multi)
                                  (car/lrem list 1 e)
                                  (car/rpush list e)
                                  (car/exec)))

but it does not leverage the use of attempting multiple times if something goes wrong/redis throws an error which atomic does. Is there a way I can send multiple atomic txns as pipeline, leveraging the features of what atomic provides?

[ptaoussanis]

@alishamohanty Hi Alisha,

I am currently working on a project that needs to send multiple atomic txs as a pipeline.

Could you give a little more context for this part - what makes you say that you need multiple atomic txs as a pipeline? What exactly are you trying to do?

As you mention, car/atomic just uses multi+exec underneath - and the problem with those is that at some point you'll need to check the exec result to see whether the transaction has succeeded.

If tx2 should occur only if tx1 succeeded, then you can't avoid checking the exec result in between - which'd mean either a roundtrip (as atomic does it), or using Lua or a module.

The details will depend on the precise semantics you need, which'll depend on exactly what you're trying to achieve.

But as a general rule-of-thumb, Lua is often a good default choice if you need non-trivial atomicity.

Hope that helps!

[alishamohanty]

Could you give a little more context for this part - what makes you say that you need multiple atomic txs as a pipeline? What exactly are you trying to do?

Little background:

I have a fn del-from-list-and-enqueue-front that moves an element in a list from anywhere to the front of list(atomically). This needs to be done for multiple elements.

I want to execute the above on multiple elements without the additional network cost. Sending commands for n elements as a pipeline seems the best option.

(defn del-from-list-and-enqueue-front [conn list-key element]
  (car/atomic
    conn atomic-lock-attempts
    (car/multi)
    (car/lrem list-key 1 element)
    (car/rpush list-key element)))

If tx2 should occur only if tx1 succeeded, then you can't avoid checking the exec result in between - which'd mean either a roundtrip (as atomic does it), or using Lua or a module.

In my usecase, tx1 and tx2 operate on different elements on same list. In short I want do multiple txs on a list that are sort of independent of each other but in a way that saves the network cost and support retries (as fn atomic).

Let me know, if you want any additional details for the problem statement.

[ptaoussanis]

There's a fair bit that's still unclear to me so there may be other alternatives too - but I'd suggest a decent starting point would be looking at a Lua script since that'll let you get whatever semantics you prefer 👍