Bootstrapping bundle

[florianlenz]

As discussed here: https://github.com/libp2p/go-libp2p/pull/302 we create a bootstrapping bundle.

General design/requirements:

1. Take in a go-libp2p-net Network, a list of bootstrap peers, and a minimum peer count. While running, monitor the number of peers we're connected to. If we drop below some number, automatically start bootstrapping. We'd probably want to listen to connect/disconnect events on the Network to do this.
2. If bootstrapping fails, backoff (probably some kind of bounded exponential backoff).
3. When we notice that we're having bootstrapping issues, periodically (e.g., once a minute?) poll network.InterfaceAddresses(). If we notice a change, try bootstrapping.

[florianlenz]

@Stebalien Could you please publish a new release for go-libp2p-net? I would like to use this file which is not included the current release.

[Stebalien]

@florianlenz all done.

[florianlenz]

@Stebalien Thank you. I used to test with a host.Host object but switched to a net.Network object like described in the requirements. Now I am getting an error when I try to dial (dial attempt failed: failed to dial <peer.ID aCpDMG> (default failure)) this happen's immediately within ~ 1/10 second. I started to check what happens in the Connect function of the basic host and I think the resolveAddrs function is probably the missing pice in the puzzle to get it work. My question is, should I really use a net.Network object or should I instead use a host.Host object? I fear that I would have do duplicate a lot of the resolveAddrs function logic and some other part's from the Connect logic. What do you think? I could also keep it flexible with a function that is called here instead of the dial method. The function would be called synchronous and could be specified by a consumer of the package in case someone does not use libp2p and would like to customize the dial process.

[Stebalien]

Hm. That's unfortunate. I'd just build on-top of the Host object for now. We can reorganize later.

[florianlenz]

@Stebalien I am getting a random error:

panic: failed to specify addrs: [/ip4/0.0.0.0/tcp/58909]

goroutine 327 [running]:
github.com/florianlenz/go-libp2p-bootstrap.(*Bootstrap).networkInterfaceListener.func1(0xc4200a6640, 0xc4204679e0)
    /Users/florian/projects/go/src/github.com/florianlenz/go-libp2p-bootstrap/bootstrap.go:81 +0x14e
created by github.com/florianlenz/go-libp2p-bootstrap.(*Bootstrap).networkInterfaceListener
    /Users/florian/projects/go/src/github.com/florianlenz/go-libp2p-bootstrap/bootstrap.go:74 +0x104
exit status 2
FAIL    github.com/florianlenz/go-libp2p-bootstrap  89.956s

which is from the go-addr-util. I don't know how to fix it / what is causing it. It happen's to me after turning on my wifi (which I turn on / off for testing). Do you know what is causing it?

I am close to done (just need to add more test's etc). Maybe you can have a look at the package and give me feedback about changes you want me to do.

Btw, do you have a prefered way of logging in the libp2p ecosystem? If so I would like to include it.

[Stebalien]

It happen's to me after turning on my wifi (which I turn on / off for testing).

It happens when you turn your wifi ON? That's really strange. The error is basically saying that you have no non-link-local IPv4 addresses (not even a loopback) but that makes no sense. Could you enable debug logging?:

1. Import github.com/ipfs/go-log.
2. Add func init() { logging.SetDebugLogging() }

Also, what operating system are you on? If you're using Linux, could you run ip addr. If MacOS, try ifconfig.

Does ipfs work in that state?

Btw, do you have a prefered way of logging in the libp2p ecosystem? If so I would like to include it.

We use go-log.

[florianlenz]

Yes, when I changed the wifi "state" (switch from on to off and vise versa). I enabled the logging but I couldn't trigger the panic again (will debug it in case I can figure out what triggers it). Ipfs work's on my machine.

[Stebalien]

Hm. I wonder if this is some kind of transient issue where we should retry....

[florianlenz]

I already retried ~50 times but couldn't trigger it.

[Stebalien]

Ah. It's going to be a Heisenbug... Well, if that function fails in production, we'll log instead of panicing so it won't be the end of the world. We'll track it down eventually.

[florianlenz]

ok. I tried the bootstrapping package yesterday in a train (our's are famous flaky wifi) and the bootstrapping worked quite well. At the moment you just call start which start the bootstrapping process. When the bootstrapping fails a "networkListener" is installed that polls InterfaceListenAddresses and wait's for a delta on the returned addresses. If there is an delta then the bootstrapping process is started again. The listener is also registered if we drop below x connected peer's. I am not sure if the "waiting of delta" option is the best since I could be connected to a wifi router, but the wifi router could be disconnected from the www. In that case there is no delta in the addresses returned by InterfaceListenAddresses as far as I could figure out. So maybe instead of polling InterfaceListenAddresses I should try to dial to a randomly selected peer from the bootstrap peer list.

[Stebalien]

Yeah, you're probably right. My intent was to avoid doing too much work in the background while still connecting the moment we notice we have an internet connection. Additionally, we might run into trouble with our backoff system if we keep trying to dial but repeatedly fail (see https://github.com/libp2p/go-libp2p/issues/1554).

Given that, the best solution is probably to:

1. Try dialing every 5m (1m?).
2. Poll InterfaceListenAddresses every 10s (?).
3. Expose a function for triggering a bootstrap event (to be used by, e.g., the DHT).

Sound reasonable? I'm open to alternative proposals if you have any.

[florianlenz]

I don't really have an alternative. A hack would be to ping a website I guess. But that's pretty ugly, so no better ideas from my side.

1. and 2. can be customized when calling the start function
2. How exactly should this look like? should it just be a function that you can call on the bootstrap struct to bootstrap manually?

[Stebalien]

1. I'd just set the parameters on init instead of start (allows us to setup/configure in one step and then start everything in the next).
2. Yes. Just a simple Bootstrap() method would work.

[florianlenz]

Ok, updated.

[florianlenz]

@Stebalien What are the next task's? I tested the bundle and it's working as it's supposed todo. Do you want to pull it in the libp2p github organisation or should I just leave it in my github account? I don't mind to hand it over to libp2p :)

[whyrusleeping]

@florianlenz probably helps for discoverability to transfer it to the libp2p org. We should get some code review in on it, and then hopefully try and use it in go-ipfs (and others) :)

[Stebalien]

(we've done one round of review, I just need to spend some time to do additional rounds)

[florianlenz]

I appreciate every code review. Just wait a few days since I am currently updating it (I will ping you when it's done)

[florianlenz]

@Stebalien @whyrusleeping I just finished the refactoring. https://github.com/florianlenz/go-libp2p-bootstrap/pull/3 I granted you write access.

[marten-seemann]

Closing, as there hasn't been any activity for almost 5 years.