mediachain / oldchain-client

[DEPRECATED] old mediachain client experiments
MIT License
4 stars 2 forks source link

Support catching up until we hit a known block #88

Closed yusefnapora closed 8 years ago

yusefnapora commented 8 years ago

This refactors the BlockchainFollower to support bailing out of the catchup process early if we hit a known block. If you pass in last_known_block_ref, the catchup process will complete once it hits that block, and you'll only receive events for the subsequent blocks.

Also changes the internals so that the catchup thread doesn't exit once it's complete. This lets us re-start the catchup if we fail mid-stream, to catch blocks we may have missed in the downtime. If that happens, it will seek back to the last seen block.

TODO:

parkan commented 8 years ago

Gonna look in the AM

yusefnapora commented 8 years ago

Last commit changes the output of TransactorClient.canonical_stream from a tuple of canonical_id, object to a dictionary:

{
  'canonical_id': ref,
  'prev_block_ref': last_seen_block,
  'record': reader.get_object(self, ref)
}

In the current transactor design, we can only know the previous block for certain, since the transactor streams events from the uncommitted block. In the next design, we'll only get block events after they're published, so we can change this to have the ref to the block in which the event occurred.

This is a breaking api change and we'll need to update the indexer. @autoencoder, does that output format look alright? Also, you probably have a better feel for where to keep track of the last seen block on the indexer side.

parkan commented 8 years ago

Let's bump version

yusefnapora commented 8 years ago

forgot to push the version bump :)