ssbc / go-ssb

Go implementation of ssb (work in progress!)
https://scuttlebutt.nz
163 stars 23 forks source link

Indexing of nulled log entries #270

Open KyleMaas opened 1 year ago

KyleMaas commented 1 year ago

See the failure in this test run while trying to track down #268:

https://github.com/ssbc/go-ssb/actions/runs/3744726663/jobs/6358410184#step:9:406

What that means is that, in the absence of anything to prevent race conditions between operations on the main log and operations on the indexes, the indexes need to be able to deal with messages which have already been nulled from the main log since messages may be nulled before all of the concurrent index processing is able to process all of the messages.

If we had some way to wait for all the indexes to catch up (see #251), then you could instead make nulling operations wait for indexes to catch up first. But in the absence of that, the indexes need to deal with this.