Open ognots opened 1 year ago
Design 1:
the first prototype of this is working successfully in theory, in https://github.com/filecoin-project/filecoin-chain-archiver/pull/67 there are however a few relevant blockers:
ExportRangeInternal
method curently returns an EOF error. this is reproduceable with lotus CLI and using filecoin-chain-archiver create
Problem Statement: Using the Lotus API, snapshot production is slow due to the constraints of Lotus's API. The average snapshot job takes 4h on average, post FVM upgrade.
Possible Solution: Reports from other operators shows snapshot production being exponentially faster writing snapshots directly to disk. Many report <1h snapshot production. Draft a design doc and prototype of a "sidecar" design, where filecoin-chain-archiver runs in a process with read-only access to a volume where Lotus can write snapshots directly to.