Controld not starting due no private IP adress found

[bbspike]

Gladius-controld cannot start on Ubuntu VPS with 2 Venets and IP adresses,

panic: Failed to get final advertise address: No private IP address found, and explicit IP not provided

goroutine 1 [running]: github.com/gladiusio/gladius-node/vendor/github.com/gladiusio/gladius-controld/pkg/p2p/peer.New(0xc42000e0b8, 0xc93d60) /ext-go/3/src/github.com/gladiusio/gladius-node/vendor/github.com/gladiusio/gladius-controld/pkg/p2p/peer/peer.go:43 +0x3e7 github.com/gladiusio/gladius-node/vendor/github.com/gladiusio/gladius-controld/pkg/routing.AppendP2PEndPoints(0xc42036a9a0, 0xc42000e0b8, 0xc42039a000, 0x0) /ext-go/3/src/github.com/gladiusio/gladius-node/vendor/github.com/gladiusio/gladius-controld/pkg/routing/routing.go:55 +0x40 github.com/gladiusio/gladius-node/vendor/github.com/gladiusio/gladius-controld/pkg/config.NodeRouter(0x0) /ext-go/3/src/github.com/gladiusio/gladius-node/vendor/github.com/gladiusio/gladius-controld/pkg/config/config.go:193 +0x4c github.com/gladiusio/gladius-node/vendor/github.com/gladiusio/gladius-controld/controld.InitializeNodeManager() /ext-go/3/src/github.com/gladiusio/gladius-node/vendor/github.com/gladiusio/gladius-controld/controld/controld.go:67 +0xa0 main.main() /ext-go/3/src/github.com/gladiusio/gladius-node/cmd/gladius-controld/main.go:8 +0x20

[revidee]

hashicorp's go-sockaddr package is the problem. In its GetPrivateInterfaces() method (ifsaddrs.go) it fetches all RFC 6890 ips.

In order to not get error we have to "fake" an RFC 6890 ip with a dummy interface. THIS IS JUST A WORKAROUND

$ ip link add eth_dummy type dummy
$ ip addr add 192.168.1.1/24 dev eth_dummy
$ ip link set eth_dummy up

Then the node can find this dummy ip, but opens TCP tunnel on 0.0.0.0, so the dummy interface should not be a problem - haven't tested thoroughly though

Edit:

It seems with this fix you won't be able to ping any other nodes

Edit2:

For the devs, the method which causes this behavior is:

/serf/vendor/github.com/hashicorp/go-sockaddr/ifaddrs.go:305

As you can see in the comments of that method, it internally does

$ sockaddr eval -r '{{GetAllInterfaces | include "type" "ip" | include "flags" "forwardable" | include "flags" "up" | sort "default,type,size" | include "RFC" "6890" }}'

include "RFC" "6890" is the problem. Installing sockaddr and executing this command WITHOUT | include "RFC" "6890" it returns the correct interface with the correct ip.

You may consider another option to retrieve the private ip.

[bbspike]

Some providers of vps does not allow add ip link. Just adding the address to LO also works.

[revidee]

ip address add 192.168.1.1/24 dev enp2s0

where enp2s0 is your interface (look it up with ifconfig) does the trick as well, nice find.

[code-merc]

I'll take a look at this when I'm in tomorrow. Think a config override for the bind address should be a quick fix, I'll create a PR on the memberlist repo with the fix too.

[code-merc]

This commit allows you to configure the bind address. It's in develop right now. You can quickly test with the environment variable CONTROLD_P2P_BINDADDRESS=AddrHere

[bbspike]

If this is also possible for networkd you can run multiple nodes on a single server with multiple NICs and dedicated ip addresses.

[broken-trigger]

Any update on this one? Its still a problem for some non private subnet users and not anyone wants or could use the additional ip for their host system.