search

filecoin-project / dagstore

a sharded store to hold large IPLD graphs efficiently, packaged as location-transparent attachable CAR files, with mechanical sympathy
Other
42 stars 24 forks source link

feat: `io.Reader` factory for `ShardAccessor` #155

Open distractedm1nd opened 1 year ago

distractedm1nd commented 1 year ago

At celestia-node, we don't only use ShardAccessor's Blockstore to serve individual blocks, but also use the underlying mount to stream the CAR file to other full nodes. We do this by adding a Reader() io.Reader method to ShardAccessor that creates independent mmap-backed (when possible) readers. We would love to upstream it to avoid maintaining a fork.

Wondertan commented 1 year ago

Crossposting my review https://github.com/celestiaorg/dagstore/pull/2#discussion_r1155812111

also cc @raulk