search

bltavares / colmeia

Attempt to make an interop layer to connect to dat on hyperswarm in Rust
24 stars 4 forks source link

WIP - Hypercore Clone & Sync - With spawn/channels #21

Closed bltavares closed 4 years ago

bltavares commented 4 years ago

To compare the approach with #20, this PR rewrites the whole architecture to spawn background tasks and communicate data upstream using channels instead of build adapters around streams.

Closes #20

bltavares commented 4 years ago

Clones a metadata and content feeds from a hypercore-daemon peer given a hash and ip:port

https://gist.github.com/bltavares/f37af663cb6f9fb4bd30ccabb1453a2a

bltavares commented 4 years ago

This design has been validated to work with a daemon running replication in the background:

image

image

The relevant line is the colmeiad service, that starts a tide webserver and expose the hyperdrive feed block info over json, while the replication task runs in the background.

https://github.com/bltavares/colmeia/pull/21/files#diff-60f4d526c84e9c5bcfc049b851ba7891R69-R89

That is quite promising, and I think this is proving to be a better design that #20

Now we have to do some cleanup and try to find a better way to manage the internal state machine, which is the core of the replication logic, without having a 300-line long match.

I would also like to test if the service would be able to provide a single listener for all background tasks, so we could have multiple hyperdrives being served from a single hyperstack