ipfs / helia

An implementation of IPFS in TypeScript
https://helia.io
Other
947 stars 106 forks source link

Multiple/alternative data retrieval implemenations #98

Open achingbrain opened 1 year ago

achingbrain commented 1 year ago

Currently Helia takes a blockstore that it enhances with bitswap. This creates a hard dependency on bitswap.

To enable experimentation and adoption of faster/more use-case specific retrieval protocols (cars, graphsync, XYZNewFutureProtocol etc) we should allow this to be a configuration option.

At this point blocks may not be the correct abstraction since it limits us to a block as the unit of data you get in response to a CID.

A better read abstraction might be a CID to a stream of Uint8Arrays? The the underlying retrieval method can apply whatever optimisations it can to fetch the data quickly and the calling code doesn't have to keep going back to fetch another block for another CID.

interface Options {
  offset?: number
  length?: number
}

interface ContentReader {
  get (cid: CID, options: Options): AsyncGenerator<Uint8Array>
}

Questions:

SgtPooki commented 1 year ago

@achingbrain Can we call this done since we completed the implementation of the BlockBroker interfaces?