Mount extensions on channel, support mux on top of corestore & corestore-swarm-networker

[Frando]

Currently the mux extensions are mounted on the main hypercore-protocol stream and not on a channel. This is why multifeed has to "own" the protocol stream (you can't pass in an existing stream into multifeed's replicate method at the moment).

I wanted to look how to use multifeed on top of corestore and with corestore-swarm-networking because this is what the dat-sdk uses, also hyperdrive needs corestore and we use it in sonar too.

The main challenge is that with hyperswarm, you don't know for which topic an incoming connection is opened. Therefore, we first open the channel with the encryptionKey (multifeed root key). Onto this mux channel the extensions are mounted.

All tests pass. One regression test had to be altered: When replicating with wrong keys, this is not an error but a timeout now. This might be intended or not, I'm not sure. Basically it waits until the other end will open a channel on the mux root key you passed. Before, it would wait for the first channel opened and error out if its the wrong one. This doesn't work in this model because there might be legitimate other channels. This will also allow to replicate multiple multifeeds over the same protocol stream! Like all shared cabals can sync on one stream between two peers.

This PR also adds a corestore.js export that has a wrapper around the mux protocol without anything else in multifeed. See test/corestore.js for now - it makes it really straightfoward to use the mux protocol with corestore and corestore-swarm-networker.

I also added a test that replicates a multifeed into a corestore+mux over hyperswarm.

I don't know if I have time for this much more. I wrote most of this some time back but never published. @kyphae and @okdistribute both recently expressed interest in this so I went over it a little and share it here now.

This PR also updates hypercore to v9, otherwise multifeed-corestore sync wouldn't work because of handshake changes. This of course likely should be a seperate patch. On the other hand, this PR would combine two wire protocol breaking changes, and with these make multifeed fully compatible with how things are done in dat-sdk and hyperdrive-daemon.

[okdistribute]

+1 to 'all shared cabals can sync on one stream between two peers.'

[m4gpi]

great!!! :boom:

[Frando]

So I've been continuing a bit on this: I have a version of multifeed that runs on top of corestore. It uses a hypercore to locally store the list of feeds that make up a multifeed, uses corestore namespaces, and connects the muxer via corestore-swarm-networking.

See my fork @frando/corestore-multifeed. With the changes in this PR, which are unrelated to adding any corestore related code, multifeed would be fully network compatible with this corestore-multifeed module.

Therefore I'm reducing the scope of this PR to just the actual change: Mount the extensions on a channel.

The question of whether the "default" multifeed model would use corestore or not can be a seperate question then. For that, I'd be interested to know what current multifeed users are thinking about using corestore for storage?

Independently of that I think the protocol change in this PR makes sense on its own, and allows for replicating many multifeeds over the same stream, and allows to be compatible to the @frando/corestore-multifeed approach.

[hackergrrl]

Looks good to me. :+1:

[DougAnderson444]

Hey gang! Any update on when this might get merged? I've been using this fork with pretty good success with a couple small tweaks.

@Frando and @kyphae I did make a few very small changes PR'd on gitlab (https://gitlab.com/coboxcoop/multifeed/-/merge_requests) that should be merged before this branch gets merged here too. Not sure who is pulling in changes over there.

[hackergrrl]

@DougAnderson444 Still remaining:

• [ ] rebase & resolve conflicts (anyone)
• [ ] merge + publish (anyone with kappa-db ownership)