We need to detect re-orgs. Each time re query for a block we should have a reference for what we expect the parent hash to be. A re-org is dectected when the retrieved block's parent hash does not match the expected parent hash.
Next we need to add some metadata to how the Exfiltrator sends data to the Loader. Instead of passing raw blocks we should do something like this:
class ChainSegment(NamedTuple):
blocks: Tuple[Block, ...]
is_reorg: bool = False
For the normal case, the exfiltrator would transmit ChainSegment(blocks=(next_block,), is_reorg=False).
In the case that a reorg has been encountered it would trace backwards up the parent_hash links until it encounters a previously known block. The exfiltrator would transmit ChainSegment(blocks=new_chain_segment, is_reorg=True). We can bound re-org detection to a fixed maximum size window and error out if tracing backwards up the chain exceeds this limit.
The is_reorg flag may not be strictly necessary but it seems better than relying on the Loader to do re-org detection.
We need to detect re-orgs. Each time re query for a block we should have a reference for what we expect the parent hash to be. A re-org is dectected when the retrieved block's parent hash does not match the expected parent hash. Next we need to add some metadata to how the
Exfiltratorsends data to theLoader. Instead of passing raw blocks we should do something like this:For the normal case, the exfiltrator would transmit
ChainSegment(blocks=(next_block,), is_reorg=False).In the case that a reorg has been encountered it would trace backwards up the
parent_hashlinks until it encounters a previously known block. The exfiltrator would transmitChainSegment(blocks=new_chain_segment, is_reorg=True). We can bound re-org detection to a fixed maximum size window and error out if tracing backwards up the chain exceeds this limit.The
is_reorgflag may not be strictly necessary but it seems better than relying on theLoaderto do re-org detection.