search

RangerMauve / hypercore-fetch

Implementation of Fetch that uses the Hyper SDK for loading p2p content
MIT License
36 stars 12 forks source link

Add `X-blocks` and `X-blocks-downloaded` headers #53

Open josephmturner opened 1 year ago

josephmturner commented 1 year ago

Thank you!

RangerMauve commented 1 year ago

Just looking into the APIs for hypercore/hyperdrive and it's all changed.

Mind elaborating on what the use case you have for this is? Is it just to see what percentage of the content has been downloaded?

josephmturner commented 1 year ago

Since for now we're not going to take advantage of sparse replication in hyperdrive.el, we can just use Content-Length.

josephmturner commented 1 year ago

Beyond X-blocks and X-blocks-downloaded headers, is it possible to send a HEAD request to determine whether a file has been downloaded already?

RangerMauve commented 1 year ago

Some thoughts:

This would add a non-trivial amount of overhead to each request since we would need to 1. get the stat for the file, 2. get the block ranges from the stat, 3. query the blobs hypercore for how much of the data is present (which might end up touching the disk), and 4. send that back serialized.

It might be good to add a header in there to opt-into getting this extra info. Maybe something like X-Show-Downloaded: true in the request headers

josephmturner commented 10 months ago

IIUC, we could use drive.blobs.core.contiguousLength once it's implemented (see https://github.com/holepunchto/hypercore/pull/378)