Adding a peer to the main cidr

tonarino / innernet

A private network system that uses WireGuard under the hood.

https://blog.tonari.no/introducing-innernet

MIT License

4.94k stars 184 forks source link

Adding a peer to the main cidr #182

Open bsamadi opened 2 years ago

bsamadi commented 2 years ago

I tried adding a peer without adding a child cidr but I got an error message like "empty list". Is it possible to add a peer to the main cidr?

alerque commented 2 years ago

I'm not sure I'm qualified to answer whether this is possible, but based on my understanding of the docs and from some real world usage it's not clear to me why you would want this. The "main" CIDR you are asking about seems to be the orchestration server ... which itself is deliberately not of any of the CIDRs. And as far as I know there can only be one such orchestration server, but itself is not meant to function as part of the WG network. You can't even set the peer name. If anything it seems to be designed to run on a host you do not want to be part of a CIDR.

What use case do you have in mind?

bsamadi commented 2 years ago

Thank you @alerque for your response. I am trying to run a k3s cluster on three VPS servers. I was hoping I could reuse one of them as both an orchestration server and a k3s server. I did it using netmaker and WireGuard.

The Netmaker version worked fine but I keep losing access to the Netmaker dashboard (port 8082), which is not needed for the network to work. I just need it to add or remove nodes. I tried running Netmaker and k3s servers on the same server and different servers. They both worked but later I lost connection to one of the nodes and the dashboard.
The WireGuard version seemed fine at first but when I deploy applications to k3s, I get some pending pods. There is definitely some WireGuard configuration that I have missed. I ran the WireGuard and k3s servers on different nodes.

I think I need to add a fourth VPS as the WireGuard orchestration server.

alerque commented 2 years ago

If your 3 VPS servers are stable (have stable IPs you can reach them on) it might actually make sense to just "hard code" your own wireguard configuration and just put them all in the same subnet yourself. For 3 pears with at least one of them having a stable way to reach it this is pretty easy to setup. Things like innernet facilitate adding and removing peers and other fancy bits, but don't add anything for the use case you describe.

If none of your 3 servers have any stable way to reach them from the other 2 then no matter what system you use you're going to want a 4th location that is stable.

To my knowledge there isn't a good way to bootstrap WireGuard or any other VPN system in a fully mesh configuration without knowing how to reach any other peers except if you can scan the local network (i.e. you can do it for disconnected peer-to-peer networks or in a LAN, but not across the public internet or on a private network that you have limited permissions to scan).

bsamadi commented 2 years ago

All of my servers have public IPs. I think it should be possible to create a virtual network on the three servers and use it to create a k3s cluster. I have found this: WireGuard setup for Kubernetes (not specifically k3s).

I wasn't sure about AllowedIPs. There are definitely other configurations that I don't know. I have not found a tutorial for creating a k3s cluster on a WireGuard virtual network.

mcginty commented 2 years ago

Hey @bsamadi - the way CIDRs work, a CIDR can either contain other child CIDRs or child peers, but not both. Since the root CIDR already comes with a sub-CIDR for the server, it cannot contain peers. You must first create a new CIDR, and put the peers inside that.

I'm curious if there are any ideas to make this "first CIDR and peer" situation more clear and usable. I definitely get how it's unintuitive what to do right away after installing the server, even with the existing messaging.

Like @alerque said, I'd also recommend if you have a static network between servers that are publicly accessible and there are only small number of peers, it's probably easiest to just use WireGuard directly rather than fuss with running an innernet server.

I have no experience with kubernetes, so can't help with anything specific to how it handles networking there.

bsamadi commented 2 years ago

Thank you @mcginty. I will try WireGuard. However, is it possible to leave it up to the Innernet user to add a peer or a sub-CIDR to the root CIDR based on what they need to do?

mcginty commented 2 years ago

@bsamadi any peer who is marked as an admin can add peers and sub-CIDRs to the network - is that what you were asking?

bsamadi commented 2 years ago

@mcginty, in my case, I don't need this feature and I don't mind it if it is there.