Sync from newest to oldest

[jbearer]

It would be better to run proactive scans, especially major scans, backwards, because the newest data is both the most likely to be missing (in the case where a node has just been offline for a short time, like an update) and the most likely to be queried.

Adding a reverse block stream might help simplify some of the transaction streaming stuff for the explorer as well.

[jbearer]

I'm thinking of an idea that might be better. Instead of having major and minor scans that run repeatedly, we just have 2 scans total, that both run perpetually, and we dovetail them in chunks. I will call these scans "heads" and "tails".

The heads scan always follows the chain head. It's job is to fetch recent blocks that we may have missed on decide, such as if we briefly lost network connection and missed a decide. The tails scan starts at the tail of the chain and follows behind the heads scan. Its job is to fetch old data that we're missing, such as after a long restart.

Both of these scans follow the same exact pattern, except for where they start and what they do when the reach the current block height:

• The tails scan starts at block 0 (or the pruned height)
• The heads scan starts at the last saved block height
• When a scan runs, it starts from where it previously left off, and fetches blocks until it reaches a chunk limit, or the tip of the chain. Then it sleeps and yields to the other scan.
• When the heads scan hits the tip of the chain, it just yields. This way it will most of the time be following very close to the chain tip, but if the chain tip ever jumps forward a lot (such as after a network outage or restart) the heads scan will go over all the blocks that were missed
• When the tails scan catches up to the heads scan (it will always catch up, because it can scan as fast as it is able, while the heads scan is bounded by the time it takes for new blocks to be decided) it starts over from 0 (or the pruned height)

The big advantage of this scheme is a bounded amount of work done each time a scan runs. We don't have occasional long pauses where a major scan runs, CPU usage spikes, and we stop fetching more recent missing blocks (because minor scans have to wait for the major scan to finish).

Another big advantage is on startup, we don't have to wait for a major scan to read from the database all the way from 0, before we start fetching more recent blocks which are more likely to be missing.