Have Example Storage Module make sure Ordering is preserved

customer: performer: deadline: estimated: started: actual: completed: confirmed: dependencies:

[ ] supersedes:

Currently in our codebase, if we run any worker example twice we encounter an issue

iex(mariari@Gensokyo)2> EClient.storage_423_from_cli("bobo")

17:25:17.292 [error] Failed to create a listen socket: :eaddrinuse.
1
2
1
{%Anoma.Node{
   dumper: %RID{id: %Pub{encrypt: 0x66a32d..., sign: 0xf2c6ee...} MOD: Anoma.Node.Dumper},
   storage_topic: %Pub{encrypt: 0x10218d..., sign: 0x7b6712...},
   configuration: %RID{id: %Pub{encrypt: 0xbdc55f..., sign: 0x416f1f...} MOD: Anoma.Node.Configuration},
   storage: %RID{id: %Pub{encrypt: 0x3cf55d..., sign: 0x0f796b...} MOD: Anoma.Node.Storage},
   logger_topic: %Pub{encrypt: 0x44636b..., sign: 0xcd48c0...},
   logger: %RID{id: %Pub{encrypt: 0xa2779d..., sign: 0x7d7df6...} MOD: Anoma.Node.Logger},
   clock: %RID{id: %Pub{encrypt: 0x5c5f41..., sign: 0x1800c7...} MOD: Anoma.Node.Clock},
   pinger: %RID{id: %Pub{encrypt: 0xf2910b..., sign: 0xae6aa2...} MOD: Anoma.Node.Pinger},
   mempool_topic: %Pub{encrypt: 0xc69945..., sign: 0xedd6d1...},
   mempool: %RID{id: %Pub{encrypt: 0xc4a882..., sign: 0x4d4938...} MOD: Anoma.Node.Mempool},
   executor_topic: %Pub{encrypt: 0x82c925..., sign: 0xb136d9...},
   executor: %RID{id: %Pub{encrypt: 0xfd7dfe..., sign: 0xa5bb7b...} MOD: Anoma.Node.Executor},
   ordering: %RID{id: %Pub{encrypt: 0x59e1ae..., sign: 0x53f3da...} MOD: Anoma.Node.Ordering},
   transport: %RID{id: %Pub{encrypt: 0xcd867e..., sign: 0x6f467f...} MOD: Anoma.Node.Transport},
   router: %RID{id: %Pub{encrypt: 0x8ea0bc..., sign: 0x90bbda...} MOD: Anoma.Node.Router}
 }, {:unix, "/home/taichi/.local/share/anoma/KLCB46UCBFMSW===.sock"},
 %RID{id: %Pub{encrypt: 0xfec3b3..., sign: 0xa20385...} MOD: Anoma.Cli.Client}}
iex(mariari@Gensokyo)3> EClient.storage_423_from_cli("bobo")
** (ExUnit.AssertionError) 

Assertion failed, no matching message after 5000ms
     The following variables were pinned:
       worker_addr = %RID{id: %Pub{encrypt: 0xfc4038..., sign: 0xa0089b...} MOD: Anoma.Node.Executor.Worker}
     Showing 10 of 12 messages in the mailbox

This is because although our example makes sure storage is ready, it does not reset the invariant on the ordering state which causes an issue when re-running examples.

I suggest we for the time being tell ordering the right spot to read at in the storage tests, so this doesn't occur.

This will be improved once #716 is done, as this could potentially fix our bad system.

anoma / green

Have Example Storage Module make sure Ordering is preserved #757