ws to port 3472 issue

[KrishnaPG]

It is not clear why the code is trying to connect to port 3472.

Testing a sample code below:

    const myCore = Hypercore(null, {
      valueEncoding: 'json',
      persist: false,
      storage: null  // storage: RAI
    })
    myCore.append(JSON.stringify({
      name: 'Alice'
    }), () => { 
      const discoveryCoreKey = 'dat://bee80ff3a4ee5e727dc44197cb9d25bf8f19d50b0f3ad2984cfe5b7d14e75de7'
      const discoveryCore = new Hypercore(discoveryCoreKey, {
        persist: false,
        extensions: ['discovery']
      })

      // When you find a new peer, tell them about your core
      discoveryCore.on('peer-add', (peer) => {
        console.log('Got a peer!')
        peer.extension('discovery', myCore.key)
      })
      discoveryCore.on('extension', (type, message) => {
        console.log('Got extension message', type, message)
        if (type !== 'discovery') return
        discoveryCore.close()

        const otherCore = new Hypercore(message, {
          valueEncoding: 'json',
          persist: false
        })
        otherCore.get(0, console.log)
      })
    });

In the browser, getting the below error:

WebSocket connection to 'ws://localhost:3472/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED

as well as this error:

[RangerMauve]

Sorry! The docs aren't super good for that. It tries to connect to that port in case you have a local gateway running for discovery-swarm-web. Would you be interested in submitting a PR for better docs or a better error message?

[KrishnaPG]

Thank you @RangerMauve . If I run the @geut/discovery-swarm-webrtc on 3472 it gives the below error now:

WebSocket connection to 'ws://localhost:3472/' failed: Connection closed before receiving a handshake response

When you say gateway you meant, the @geut/discovery-swarm-webrtc right?

Also, observed that the discovery is kind of segregated:

• web peers only discover web peers
• node native peers only discover native peers

Was hoping that same code when run on web and native (nodejs instance), both will discover each other. Is there a way to make that kind of interoperable discovery work, across web and native instances?

May be the native discovery (hyper-discovery ?) should include the web signaling inside itself, so that

1. we do not need to run separate instance of gateway
2. it can bridge the native and web discovery peers seamlessly,

[e-e-e]

Would it make sense to make this optional rather than the default. As a user I do not expect a package to automatically search for local websockets without me having first created them and then explicitly asked for them to be connected too.

Is there a specific reason to have this as the default behaviour?

[RangerMauve]

@KrishnaPG The gateway you'd want to run is from discovery-swarm-web discovery-swarm-webrtc runs a signaling server for making webrtc connections.

Regarding the peer discovery, there's been some trouble getting web peers to discover node peers when the node peers are running on networks that aren't easily routable from the internet. This will hopefully improve once we get Dat 2 put together and integrate hyperswarm-web into the SDK.

You don't need to run the gateway locally, the library optionally attempts to connect to it if it exists, but it should be falling back to the public gateway if there is no local one.

The reasoning I had when creating this function was to reduce the reliance on public gateways and make the library more resilient without needing people to mess around with the configuration. My goal with the SDK has been for stuff to "just work" with some powerful defaults.

Also part of the reasoning is that public gateways are super expensive to run, so anything to reduce traffic on them would help immensely. :P

[karlmoubarak]

Hey @RangerMauve , I am getting the same errors as @KrishnaPG . It doesn't seem that the library is falling back to the public gateway automatically. Is there a way around this without having to manually run the signalling server?

Thanks in advance for your help!

[okdistribute]

@karlmoubarak if you're doing active development on your own project, it's recommended to run your own signalling server as the public one is just for testing and playing around with and doesn't guarantee uptime

[karlmoubarak]

@okdistribute great, thanks!

[RangerMauve]

@karlmoubarak Have you checked the network tab to see if it's making the connection? Are you just getting the error about it not being able to connect to localhost?

The server seems to be doing fine

[karlmoubarak]

Alright, so I am doing local development where I am reading an archive from a browser (chrome).

→ When running my own signalling server, the network tab shows the traffic coming and going normally, and the data is pulled from the archive successfully and displayed in the browser as expected. → When letting it fall back to discoveryswarm.mauve.moe, the network traffic seems to be doing fine (same as running my own server), except I am getting the same error messages as above, and the data is not pulled or displayed in the browser.

I thought this might have something to do with that I am writing the archive on the same device/network, but I tested this on a different device/network/browser, and the webpage remains static (doesn't pull from the archive). Would you like to see the code?

[RangerMauve]

I think the issue there is that the swarm on mauve.moe isn't able to get past your NAT. :(

The way discovery-swarm-web works is that it can only make outgoing connections which is less reliable than getting incoming connections. So even if your computer could normally connect to the discovery server, it might not be able to receive it's connections and wont be able to replicate the archive with browser peers.

We have the new hyperswarm module and hyperswarm-web which might help out with the NAT traversal issue, but that's only being used in the latest (and slightly experimental) version of Dat which doesn't work in browsers yet (due to missing crypto libraries in JS).

If we got the hyperswarm stuff into dat-node in tandem with discovery-swarm, we could potentially modify the SDK to use the v9 version of hyperdrive paired with hyperswarm-web to see if that improves reliability. 🤷

Alternately if we got discovery-swarm-webrtc working in dat-node we could get node.js peers participating with web peers.

I don't think I'd have time for doing these things though, and neither of these would work with Beaker. 😅

It might be best to try to push the crypto stuff in Hyperdrive v10 forward and migrate everything to use the latest and greatest.

[karlmoubarak]

Yeah so... The problem was my NAT. It seems that having the webpage tested by different browsers on different networks never yielded the same result. It either works or doesn't. But I'll take what I can for now.

Anyways, thanks @RangerMauve for your support, and for the extra cool stuff being worked on with DAT. Keep it up! I'm excited to see what comes out :D

[RangerMauve]

Gonna close this for now since it's not very actionable yet