djordon commented 1 month ago

Feature - Add functionality for backfilling the Stacks blockchain

1. Description

1.1 Context & Purpose

We always need to have a complete view of the Stacks blockchain. Some events, like a signer is restarted, may result in the signer having an incomplete view so we need functionality that can make the signer's view of the canonical Stacks blockchain complete.

2. Technical Details:

2.1 Acceptance Criteria:

[x] Add a function that continues to fetch Nakamoto blocks until it finds a block that it has already seen or has fetched all Nakamoto blocks.

3. Related Issues and Pull Requests (optional):

This can be built off of the functions added in https://github.com/stacks-network/sbtc/pull/238.

netrome commented 1 month ago

I think we've had a misunderstanding here. I believe we already have backfilling logic. The signer data model is already fork-aware, so forking isn't really an event. Also restarts shouldn't really matter. The logic in the block observer is such that if the signer discovers a "missing parent" it will query for that block until it connects to a block already known to the signer.

What we need to change based on our previous conversation is that the block observer in the signer must consume a stream of stacks block hashes in addition to the bitcoin block hashes, but the rest of the logic should basically be the same afaik.

Happy to discuss this further.

djordon commented 1 month ago

The logic in the block observer is such that if the signer discovers a "missing parent" it will query for that block until it connects to a block already known to the signer.

Yeah I think we're on the same page. The process of fetching missing parents is what I call "backfilling". It's probably not the right word. This ticket is to write the code that executes the logic of fetching missing parent blocks until we get a block that we already know about.

The signer data model is already fork-aware, so forking isn't really an event.

Oh yeah right, we don't need to worry about forks.

Also restarts shouldn't really matter. The logic in the block observer is such that if the signer discovers a "missing parent" it will query for that block until it connects to a block already known to the signer.

Yup, we're on the same page. A signer restart is an example where we can miss stacks blocks from the block observer and need to fetch missing parent blocks. Stated another way, restarts may require us to "backfill" missing blocks.

I need to update the working of this ticket. The term "out-of-date" is wrong or misleading, I think I should change it to "incomplete".

stacks-network / sbtc