Enable efficient indexing of historical chain data

[fridrik01]

See https://github.com/filecoin-project/fvm-pm/issues/299

Lotus currently uses the following sqlite databases:

• sqlite/events.db: Stores events sent by actors in the FVM
• sqlite/txhash.db: Stores mappings of Eth tx hash to Filecoin message cid
• sqlite/msgindex.db: Stores block messages cid and their tipset cid for faster lookup

We should try to unify all the different databases into one which should make maintaining correctness and doing recovery/backfilling simpler.

Also, we make sure we handle the following:

• Make sure we handle all edge cases (forks, reverts, config changes that require pruning, etc)
• Allow enabling/disabling what to index
• Be able to configure lookback so these indices don't grow endlessly Not needed as the indexes are tiny compared to chain data (less than 0.25%)

• 11007

[raulk]

• populating on snapshot import if not already doing

[BigLep]

Newbie question: what was the reason for having 3 separate databases to begin with?

[Stebalien]

I believe the idea was that it made it easy to tell if one was getting large and remove/disable it. I'd kind of like to introduce (even in shed) some form of GC command before we unify them, but I do think unifying them is the way to go (that and re-organizing our tables to massively reduce the amount of duplicate data).

[Stebalien]

In terms of: why multiple observers? We wanted to keep these subsystems separate. But I wouldn't be opposed to a new architecture here (that, e.g., lets us cleanly keep track of what we've indexed and what we haven't) as long as we can make it somewhat pluggable. HOWEVER, if we want to be able to enable/disable these indices independently... we'll need to track what has been indexed and what has not been indexed independently as well.