Should ssb-config be asynchronous?

[christianbundy]

We've been hardcoding the hostname to listen on, but unfortunately that's causing a few problems:

1. We have to choose between 0.0.0.0 and :: for IPv4/IPv6.
2. Quirky environments like Travis let us bind to IPv6 but don't allow connections.
3. We want to broadcast network interface address, not a meta-address.

I've spent a few hours today working on a module that returns reachable network addresses, which solves all of the above (!), but it's asynchronous and therefore would require changes to ssb-config.

The gist is that you give it some port numbers and then it enumerates the device's network interfaces and returns only the addresses that it can connect to itself with. Even better, we can hand the non-internal addresses off to multiserver so that the UDP broadcast includes multiple addresses:

• IPv4 addresses
• IPv6 addresses
• Tor/cjdns/etc

Here's some sample code that sets config.connections.incoming dynamically:

const defaultPorts = require('./default-ports')
const rni = require('reachable-network-interfaces')

const scopes = {
  internal: ['device'],
  external: ['device', 'local', 'public']
}

const incoming = {}

rni(defaultPorts, (err, reachable) => {
  if (err) throw err

  Object.entries(reachable).forEach(reachableEntry => {
    const [ service, addresses ] = reachableEntry

    // define incoming array for service
    incoming[service] = []

    addresses.forEach(info => {
      incoming[service].push({
        host: info.address,
        port: defaultPorts[service],
        scope: info.internal ? scopes.internal : scopes.external,
        transform: 'shs'
      })
    })
  })

  console.log(JSON.stringify(incoming, null, 2))
})

And the output:

click to expand

```json { "net": [ { "host": "127.0.0.1", "port": 8008, "scope": [ "device" ], "transform": "shs" }, { "host": "::1", "port": 8008, "scope": [ "device" ], "transform": "shs" }, { "host": "192.168.3.55", "port": 8008, "scope": [ "device", "local", "public" ], "transform": "shs" }, { "host": "172.18.0.1", "port": 8008, "scope": [ "device", "local", "public" ], "transform": "shs" }, { "host": "fce2:9811:4862:81a7:bb08:91d6:2e41:d220", "port": 8008, "scope": [ "device", "local", "public" ], "transform": "shs" } ], "ws": [ { "host": "127.0.0.1", "port": 8989, "scope": [ "device" ], "transform": "shs" }, { "host": "::1", "port": 8989, "scope": [ "device" ], "transform": "shs" }, { "host": "192.168.3.55", "port": 8989, "scope": [ "device", "local", "public" ], "transform": "shs" }, { "host": "172.18.0.1", "port": 8989, "scope": [ "device", "local", "public" ], "transform": "shs" }, { "host": "fce2:9811:4862:81a7:bb08:91d6:2e41:d220", "port": 8989, "scope": [ "device", "local", "public" ], "transform": "shs" } ] } ```

Is this useful enough that we should consider switching ssb-config to an async style?

cc: @dominictarr @mixmix @arj03 @cryptix @regular

---

Just tested my module on Travis and it seems to work on all platforms, which gives us all the benefits of dynamic network interfaces without the weird breakages.

[dominictarr]

hmm, it sounds like there is a different root problem here. This is actually about detecting network settings, not about loading a configuration file. Not everything that uses ssb-config (for example the cli, or ssb-client) needs to be able to look up the network address, not unless it's starting a server. I think this is actually a separate concern. Maybe the better approach is to move that out of ssb-config? maybe ssb-config should be the part that can be synchronous?

There is a messy problem that seems to be emerging from the network address stuff, but "make ssb-config async" feels like you've half-decided what the solution is.

Currently the part that needs it reads that information from ssb-config, but maybe that is the problem?

[christianbundy]

I think you're right. The root problem seems to be that ssb-config is static rather than interactive, so we're having trouble defining properties that may change over time (e.g. reachable network addresses).

I remember reading some posts about ssb-config being interactive, but I don't remember how the discussion around that went. Is that something that you've considered/discussed in the past?

[dominictarr]

I don't remember discussing that. I am aware of one or two things that do write a config file though. Is that what you mean?

To me, "configuration" means parameters the user controls. If it's state that is read from the state of the network/system at run time, that seems like quite a different thing from, for example the number of hops to replicate.

[christianbundy]

I can't remember the thread now, but it was about how it would be useful to allow the config to change during runtime but that modules using the config currently expect it to be static.

My goal is to resolve https://github.com/ssbc/patchwork/issues/958, which seems to be breaking SSB replication on local networks when:

• ssb-config sets { host: '::' }
• somewhere our stack chooses one interface to broadcast as a multiserver address
• often the chosen interface doesn't work

Instead it seems like we should be broadcasting all addresses, which is what this PR does (although it also ensures that those addresses work). Maybe I should make a PR that sets defaults from os.networkInterfaces() synchronously instead?

[the-kenny]

Instead it seems like we should be broadcasting all addresses, which is what this PR does (although it also ensures that those addresses work). Maybe I should make a PR that sets defaults from os.networkInterfaces() synchronously instead?

I think we should broadcast the IP of each respective interface on that specific interface. But we have to consider which IP to broadcast (v4, v6, and which of the multiple v6 IPs of that interface?).

My feeling is that this should be configurable: By default it should do the "sane" thing by default and support specifying the list of interfaces and which ip is broadcasted for each interface as a fallback. People might want to only broadcast on their wifi adapter's interface and not on their VPN, for example. Another thing people might want to do is broadcast their hostname over their WAN interface and some other IP to their local network.