Expose chain syncing information

[travisperson]

Description

Currently the only way to understand what is happening during chain syncing is to read through log messages.

This is difficult and not ideal for users, it's also not convenient for testing as log messages may change frequently and makes testing for certain information fragile.

We should provide a cli command for users that will provide information to users so they can understand the current state of their chain.

There is already work being done to improve the chain syncing of the daemon at the moment. This work would tail off of the improvements being made.

There are two main pieces of information we want to know

1) Is chain syncing running? 2) What are we syncing? 3) Current progress?

To answer these questions a first proposal would to be expose the start / end time of the last / current call to HandleNewTipset, along with the tipset key.

{
   "mode": "",
   "start": "",
   "end": "",
   "tipset": [
      ""
   ],
   "delta": 0
}

key names are completely up for change

Acceptance criteria

• CLI command to expose chain syncing status
• Result contains the start to the last call of HandleNewTipset
• Result contains the end to the last call of HandleNewTipset or null if it's running
• Result contains the mode of the chain syncer (Syncing or CaughtUp).
• Result contains the tipset passed to HandleNewTipset.
• Result contains the delta, which is the block difference between the last tipset fetched and our chain height.

---

I defer strongly to suggests from @frrist who is currently leading the chain syncing rework.

[anorth]

Thanks for filing this. I think we can expose some great info about both downloading and validation, more than you've suggested, in a chain status command. @frrist can work out the details after we've finished the present refactors for graphsync and mode.

[frrist]

I think this looks reasonable with a few modification to the structure that's reported:

type SyncerStatus struct {
  // Running is true when the syncer is processing ChainInfo, false otherwise.
  Running bool
  // NewChain is the ChainInfor the syncer is either activally processing or the last
  // ChainInfo value the syncer processed.
  NewChain *types.ChainInfo
  // Started is the time when the syncer started processing Chain.
  Started time.Time
  // Trusted indicates whether or not this sync was triggered by a trusted peer
  Trusted bool
  // CurrentHead represents our nodes best verified TipSet
  CurrentHead types.TipSetKey
  // CurrentHeight represents the height of our nodes best verified TipSet
  CurrentHeight uint64
}

Given this information it should be do-able to figure out what we are syncing and our progress, and all of this information is available inside HandleNewTipSet.

[frrist]

One not so straight forward part of this- nothing about the chain syncer has been exposed over the CLI before. The canonical way to accomplish this is via plumbing/porcelain. This issue implies that the syncer will need to have its methods (at least Status, but why not all of them while we are at it) added to the plumbing API. Any objections to this?

[anorth]

Moar status!

• Since this structure is intended for external consumption, we should decouple it from the internal ChainInfo (also, please extract the trusted flag from that).
• For long fetches, include information about how much has been fetched so far (a tool could then indicate progress-%).

type SyncStatus struct {
  // The heaviest that has been fully validated
  ValidatedHead types.TipSetKey
  // The height of ValidatedHead
  ValidatedHeadHeight uint64

  // They key of the head of the chain currently being fetched/validator, or undef if none
  SyncingTip types.TipSetKey
  // The height of SyncingTIp
  SyncingHeight uint64
  // Whether SyncingTip is trusted as a head far away from the validated head
  SyncingTrusted boolean
  // The lowest height fetched for SyncingTip
  SyncingFetchedToHeight uint64
  // Whether fetching for SyncingTip has completed (reached ValidatedHead)
  SyncingFetchComplete boolean
  // The time at which SyncingTip began
  SyncStarted time.Time
}

[frrist]

Question about SyncingFetchedToHeight. I think this implies the fetcher will need to store or emit the most recent tipset it has fetched, would you expect:

• a polling method like LastTipSetFetched() types.TipSet
• a publishing channel, similar to HeadEvent on the chain store

[anorth]

I would expect a callback passed to the fetching method. It already has one, which I hope is sufficient!