Per issue #23, any error while processing a branch of the dag results in that
branch never being processed again, as we have the blocks in the blockstore and we
assume that every block we have has been processed.
This PR adds a repair goroutine that runs whenever the crdt-store is marked as dirty.
The crdt-store gets marked as dirty when errors have occurred when "processing
nodes" or "sending new jobs" (fetching nodes from the network usually).
This task, which runs every RepairInterval, starts walking down the DAG from
the heads and retriggers re-processing for every node that was not processed.
It does not stop until the bottom has been reached.
In order to know which DAG nodes have been processed and which ones haven't we
need to explicitally store the list of processed nodes (which is different
from the list of nodes that we have). A node becomes processed as soon as its
delta has been merged.
Repair can be an expensive operation thus we should avoid marking the
datastore as dirty easily, which perhaps is the case when we shutdown while
processing things.
Once repair reaches the bottom, the dirty bit is cleared.
Datastores in previous versions will reprocess the store on first run, as they
need to mark all the nodes in the DAG as processed.
Per issue #23, any error while processing a branch of the dag results in that branch never being processed again, as we have the blocks in the blockstore and we assume that every block we have has been processed.
This PR adds a repair goroutine that runs whenever the crdt-store is marked as dirty.
The crdt-store gets marked as dirty when errors have occurred when "processing nodes" or "sending new jobs" (fetching nodes from the network usually).
This task, which runs every RepairInterval, starts walking down the DAG from the heads and retriggers re-processing for every node that was not processed.
It does not stop until the bottom has been reached.
In order to know which DAG nodes have been processed and which ones haven't we need to explicitally store the list of processed nodes (which is different from the list of nodes that we have). A node becomes processed as soon as its delta has been merged.
Repair can be an expensive operation thus we should avoid marking the datastore as dirty easily, which perhaps is the case when we shutdown while processing things.
Once repair reaches the bottom, the dirty bit is cleared.
Datastores in previous versions will reprocess the store on first run, as they need to mark all the nodes in the DAG as processed.