This PR adds Pebble-based MVCC storage for Payloads.
It layouts payloads on disk using the following suffix encoding:
{Owner, "/", Key, "/", EncodedHeight}
Where EncodedHeight is the height's 1's compliment encoded in Big Endian. This way we can use SeekPrefixGE API (which, as name implies, does not support reverse iteration).
SeekPrefixGE has an advantage over the SeekGE that it uses bloom filter and reduce read IO.
Values are encoded "as is" and we rely on database block-level compression and caching.
Note to reviewers: PR looks quite big, but it is mostly MVCC tests and benchmarks, for the core code there are only ~100 LOCs in service/storage2/payload/payload.go.
This PR adds Pebble-based MVCC storage for Payloads.
It layouts payloads on disk using the following suffix encoding:
Where
EncodedHeightis the height's 1's compliment encoded in Big Endian. This way we can use SeekPrefixGE API (which, as name implies, does not support reverse iteration).SeekPrefixGEhas an advantage over theSeekGEthat it uses bloom filter and reduce read IO.Values are encoded "as is" and we rely on database block-level compression and caching.
ref: https://github.com/onflow/flow-go/pull/3937 ref: https://github.com/onflow/flow-archive/issues/99 ref: "CockroachDB MVCC".
depends on: https://github.com/onflow/flow-archive/pull/105