holepunchto / hyperdrive

Hyperdrive is a secure, real time distributed file system
Apache License 2.0
1.85k stars 136 forks source link

Use subEncoder for files sub #333

Closed HDegroote closed 1 year ago

HDegroote commented 1 year ago

Original here: https://github.com/holepunchto/hyperdrive-next/pull/57

Marked as draft because I'm not sure exactly which parts of the code related to batches were work-arounds and which were on purpose. Note in particular that I am now passing the blobs through as a parameter instead of loading it again from this.db in a batch, which enables me to use the _db arg to pass the batch and to get rid of the _files arg (I couldn't figure out why a batch needed to reload the blobs, but there might have been a reason).

There's also a TODO marked in the PR to verify something in the close logic, where we recently made a fix with comment 'workaround to flush the batches for now.' => I think that's no longer relevant, but sanity checking that before fully removing

HDegroote commented 1 year ago

Also pushed another commit to refactor watch(), so it also uses FILES_SUB instead of defining its own subEnc

HDegroote commented 1 year ago

This PR has a bug when creating a batch on a non-ready core--looking into it.

HDegroote commented 1 year ago

Fixed the bug in a not-so-elegant way, by always reloading the blobs (as before this PR)

The current flow for a batch is:

LuKks commented 1 year ago

@HDegroote Because a batch instance doesn't have getHeader that's the issue, right? The easy fix is to hack it around and use batch.tree property when it's a batch, or to add the method to the Batch class

LuKks commented 1 year ago

I moved the branch to the repo itself while preserving your commits so I could do some debugging: https://github.com/holepunchto/hyperdrive/pull/344

mafintosh commented 1 year ago

landed in main