search

gravitl / netmaker

Netmaker makes networks with WireGuard. Netmaker automates fast, secure, and distributed virtual networks.
https://netmaker.io
Other
9.5k stars 552 forks source link

Keep server and netclient separated #624

Closed shizunge closed 2 years ago

shizunge commented 2 years ago

From https://github.com/gravitl/netmaker/issues/617#issuecomment-1015559608 It mentioned that

Going forward we will likely require client mode in order to run the netmaker server (will likely take out the option to set this variable). This is because a new set of features will require having a wireguard client on the server in order to function correctly.

I am not sure what features in the road map, but I want to express my concern on coupling the server and netclient. (If you are only meant to run some wireguard command, but not to set the machine where the server is running as a wireguare peer. Then it is fine.)

See the diagram from https://netmaker.org/architecture.html, the UI and the server today are node independent, while a netclient is always associated with a node. If server also meant a netclient, it would tie the server to a node and would make it difficult to scale. For example (Especially when we want to have different setup on nodes/netclients).

  1. Migrate server from one node to another node (The node may or may not belong to the mesh).
  2. Running multiple server via high available installation.

I still believe the architecture described in https://netmaker.org/architecture.html is better: separating the server and netclients. Any change to netclient should be made via grpc.

afeiszli commented 2 years ago

There is one current feature that relies on the client: UDP Hole Punching.

UDP Hole Punching requires clients to establish an outbound connection with the server. The server then examines this connection to get the return address/port, which it shares with other peers. Without a client running with the server, we cannot run this feature.

There are two future features which will also rely on a mix of client and server: Secure Server and Metrics/Advanced Routing. Secure Server will allow all client-server comms to occur over WireGuard (no public endpoints). This requires having clients built into the server. Metrics/Advanced Routing will require examining active connections to see what should be done with the routes.

I see two alternatives that might work for you.

Alternative 1: Bare Metal Server The server is run outside of docker, which gives it direct host network access, and the client functions largely as a normal client.

Alternative 2: Co-hosted client This is something we recommend currently, which is just to run a separate netclient alongside the server. Really, the "server client" is mostly contained. The interfaces exist only inside the container. It just consumes ports on the host. So you can run a regular netclient outside of the server but on the same host.

afeiszli commented 2 years ago

We will need more information on how this would block a scaled setup. Based on your examples, I don't think it would be a blocker.

1. Migrate server from one node to another node (The node may or may not belong to the mesh). This is not typical, but could be done. I dont think anything blocks you from running the server where a current node is. The only thing that would need to change is the active node's port.

2. Running multiple server via high available installation. We already enable this, and there is no limitation on HA using the client mode. Every server just has its own set of clients.

shizunge commented 2 years ago

I believe they are all about wireguard peer to peer communication. I do not understand why they cannot be done by a netclient, getting configuration from server via grpc? In wireguard, there is only peer, and not concepts about client/server.

E.g. UDP Hole Punching, though I am not super familiar with it, but I believe it could be done:

  1. Starting server
  2. Node0 connects to server via grpc (tcp)
  3. Node1 connects to server via grpc (tcp)
  4. Node0 learns from the server it needs UDP hole punching to keep connection between Node1
  5. Node1 learns from the server it needs UPD hole punching to keep connection between Node0
afeiszli commented 2 years ago

@shizunge we are going to close this for now. It is not aligned with the roadmap of the project, and we intend to continue having server-side features that are dependent on a wireguard connection with the clients, which makes this logic necessary.