Advertise on the network.

[RangerMauve]

We have a problem with creating archives inside dat-js where even though the archive is created, no other Dat tools are able to find it.

This is due to the fact that discovery-swarm-stream is only able to search for peers and doesn't advertise itself as a peer.

I propose the following:

• Advertise for the given channels
• When you get an incoming connection
  • If no channels are joined, drop the connection
  • If only one channel is joined, break the connection and create a proxy connection to the source per peer
  • If multiple channels are joined, parse the connection through hyprcore-protocol
  • Upon getting the first discovery key, break the connection and create a proxy to the source per peer that's subscribed on that discovery key

cc @DougAnderson444 @karissa

[Frando]

Ha, there is an issue about this! Was having a hard time debugging why when working on https://github.com/Frando/corestore it would not sync properly to node even though the general swarming seemed to work. Would be great to get this fixed! :-)

[RangerMauve]

Yeah! it's next on my TODO list. I'll probably have it released early next week. :D

[DougAnderson444]

@RangerMauve Can we name change his thread to Advertise Dat and Archive version updates on the network

• [ ] Expected behavior: Update an archive version in Dat-Js, see the same version increment in dat-node
• [ ] Observed behaviour: Update a Dat archive version in Dat-Js, no version change in the dat-node listening for it, plus archive.on('sync') never fires.

I can't seem to update a version using Dat-Js, then have the new version picked up by Dat-Node instance. Seems like the first version is synced no problem, but more recent versions don't get through. archive.on('sync') never fires in dat-node either. datbase.org gets the version updates, and if I ping datbase.org,, then dat-node updates. So Dat-Js is updating databse.org, but not dat-node.

Try this test to replicate what I am seeing:

• [ ] Browser: Create a dat in a browser using Dat-Js (I'm using this one in Glitch)
• [ ] Node: Listen on that Dat using dat-node in nodejs (like this one)
• [ ] Browser: Back in the browser, edit a file in the dat, which updates the archive version
• [ ] Node: Watch for updates to the version (I'm not seeing updates)
• [ ] Browser: Load up the same dat on datbase.org.... NOW you'll see the dat-node update to the most recent version

Maybe this is in the difference in the way Dat-Js updates dat-node vs datbase.org (hasbase?)

[DougAnderson444]

More testing this morning, discovery issue #8 still exists between a new Dat-JS and a listening dat-node.

Create a new Dat with Dat-JS and try to replicate the archive in a dat-node, doesn't discover, unless you bring it up on a super peer like datbase.org...

[RangerMauve]

I think the issue of needing a third party is related to your NAT.

The server dat-js uses is only able to create outgoing connections, so when a peer is behind a NAT that can't be hole punched easily, it cannot be connected to. Hashbase is different in that it can have incoming connections, too. So your node can send a connection to it through it's NAT.

I'm not sure what's up with updates not going through though. Maybe it's missing the {live: true} flag somewhere?

[RangerMauve]

I think hyperswarm will address the first problem once it lands. I'm not sure yet about the second problem.

[DougAnderson444]

Hmm, ok @RangerMauve thanks. I would have thought that the moe dat-gateway would be able to connect to the dat-node with no issue? Awww, NATs.

Any workaround suggestions? Maybe something like:

1. Create a Dat-JS
2. Automatically register that Dat-Js hex url on a hashbase instance / super peer
3. Then try to replicate the archive using dat-node?

or maybe,

1. Create Dat-JS
2. Connect via websocket to the nodejs running the dat-node?

I can try these out once I get it all set up. Seems a bit hacky though. I'm really interested in people being able to create the Dat in the mainstream browsers (and replicate it).

[okdistribute]

dat-node doesn't currently connect to a gateway but that would be awesome. @pvh talked about this being useful too.

[RangerMauve]

You can try running a local discover-swarm-web server on your machine with the following command:

npx discovery-swarm-web

The only caveat is that you'll need to use http instead of https due to some browser restrictions when connecting to localhost.

[RangerMauve]

Otherwise, I think we'll just need to wait a bit for hyperswarm to be finished so we can integrate it. Mafintosh seems to be saying that the NAT hole punching has gotten a lot better for situations like this.