[DNM] Hash file simplification

[antiochp]

Deprecate the (P)MMR data file (and size file) and store (leaf) data in the database. This PR reworks the PMMR backend data strcutures to only store the hash file on disk. Everything else is pushed into the db (indexed by MMR pos).

Very much still an experimental WIP.

Outputs, rangeproofs and kernels -

• store entries in the db indexed by MMR pos (per leaf) pos -> output
• existing index still maps output commitment to output pos etc. commitment -> pos
• commitment -> pos -> output

Headers -

• already store headers in the db indexed by hash hash -> header
• store new index mapping height -> hash
• height -> hash -> header

---

TODO

• [ ] get chain crate tests passing
• [ ] get other tests passing
• [ ] cleanup all the commented out code
• [ ] lots of missing docs on new fns etc.
• [ ] migration process for existing node
  • [ ] populate the db on startup if missing if required
• [ ] handle receiving txhashset state sync
  • [ ] given a raw "data file" and a prune_list we can sequentially read leaf data for all unpruned leaf pos
• [ ] handle providing txhashset state sync (need to recreate data files from db etc.)
  • [ ] iterate over unpruned leaf data in db and write bytes to "data file" when constructing txhashset.zip
  • [ ] txhashset.zip is cached every 720 blocks so we can take advantage of this to mitigate extra cost here
  • [ ] consider not supporting state sync initially (no TXHASHSET_HIST capabilities)

[yeastplume]

Closed and marked as 'abandoned'. This does not mean the PR is unworthy. This means that merging it would require further work that nobody appears prepared to do at this time, or the original author has been unresponsive for a significant amount of time. Anyone who want to continue work on this functionality is free to re-open this at any time.