concurrency issue evaluation

[chicco785]

just a side note. as we talked about in the past, the general assumption when replicating data in anubis is that updates are infrequent and almost never happen concurrently.

why am I bringing this up again? well, I could be wrong, but if I understand what's happening under the bonnet (and it's a big if), then policies data replicated in OPA could be inconsistent if we bring concurrent writes into the mix. Here's the scenario

• Anubis holds policies for tenant Tenny: [p1, p2].
• p1 is Bob's policy whereas p2 is Alice's.
• Alice writes a new policy p3.
• At the same time, Bob deletes p1.

What will end up in OPA? It's possible OPA eventually contains [p2] (or even [p1, p2, p3]) instead of [p2, p3] if I interpret the code correctly. In fact, here's a possible execution history:

1. (Bob's thread) Anubis deletes p1 and then reads all of Tenny's policies, so it gets [p2].
2. (Alice's thread) Anubis writes p3 and reads all of Tenny's policies, so it gets [p2, p3].
3. (Alice's thread) Anubis PUTs [p2, p3] into OPA.
4. (Bob's thread) Anubis PUTs [p2] into OPA.

Originally posted by @c0c0n3 in https://github.com/orchestracities/anubis/issues/195#issuecomment-1383848966

[c0c0n3]

@Cerfoglg @massimoneri suggested to push policy updates in the background at regular intervals, e.g. every hour.

[c0c0n3]

closed by #209