[Bug]: Setting up Egress Client, but cannot connect to egress network.

Contact Details

cedric@verstraeten.io

What happened?

I'm having a basic setup (nothing special), but can't get my head around it. It looks as following:

Screenshot 2023-11-23 at 09 33 23

I have a local network with IP ranges: 192.168.1.0/24, which I would like to reach from another Peer (a VM in the cloud, fake2). After setting up (which was really super easy..) I'm able to use the wireguard network to ping the different peers. I then added the Egress gateway to my client kerberos, to allow proxy for 192.168.1.0/24 and 192.168.2.0/24.

interface: netmaker
  public key: +mCQBIn1ZyioxxxxxhFLsHoQ/ZL/Wo=
  private key: (hidden)
  listening port: 51821

peer: QdMxxxxxHgLk5C+xxxx/P3o3ycR4=
  endpoint: 104.xxx.xxx.150:51821
  allowed ips: 10.101.0.3/32
  latest handshake: 7 seconds ago
  transfer: 108.29 KiB received, 27.54 KiB sent
  persistent keepalive: every 20 seconds

peer: xxxx+YeWevwh0w1oxXuOAg=
  endpoint: 178.xxx.xxx.25:51821
  allowed ips: 10.101.0.2/32, 192.168.2.0/24, 192.168.1.0/24
  latest handshake: 1 minute, 35 seconds ago
  transfer: 3.07 MiB received, 820.11 KiB sent
  persistent keepalive: every 20 seconds

peer: nHd28igw4yLKxxxxx7kwincDsUxTK9I2ks=
  endpoint: 37.xxx.xxx.83:24609
  allowed ips: 10.101.255.254/32
  latest handshake: 25 minutes, 34 seconds ago
  transfer: 153.58 KiB received, 2.96 MiB sent

By doing this I'm able to ping the Egress gateway using its internal IP 192.168.1.238 from a remote peer.

root@fake2:~# ping 192.168.1.238
PING 192.168.1.238 (192.168.1.238) 56(84) bytes of data.
64 bytes from 192.168.1.238: icmp_seq=1 ttl=64 time=43.0 ms

However I cannot ping any other IP address in the local network 192.168.1.0/24. Doing a traceroute it finds out to use the Egress gateway but cannot reach it in the end.

root@fake2:~# traceroute 192.168.1.14
traceroute to 192.168.1.14 (192.168.1.14), 30 hops max, 60 byte packets
 1  10.101.0.2 (10.101.0.2)  257.297 ms  257.110 ms  257.073 ms
 2  * * *
 3  * * *
 4  * * *

Version

v0.21.2

What OS are you using?

Linux, I'm running the netmaker server on a Digitalocean machine.

Contributing guidelines

[X] Yes, I did.

I have the same problem and managed to work out what is not working, I hope this can help.

My network is like the one in the graph below

so my clients connect to VPS-1 (ingreess server) and i would like to access the 192.168.1.0/24 subnet on docker-01 (egress) This does not work.

After playing around with the wireguard settings I figured out it works (but only for a few minutes) if add the subnet to the WG allowed ips in both nodes (vps-1 and docker-01) examples below of wireguard config files I used

On docker-01

[Interface]
ListenPort = 51825
PrivateKey = ***

[Peer]
PublicKey = ***
AllowedIPs = 10.101.0.1/32, 0.0.0.0/0, 10.101.255.254/32, 10.101.255.252/32, 10.101.255.253/32, 192.168.1.0/24
Endpoint = 140.xxx.xxx.xxx:51821
PersistentKeepalive = 20

on vps-1

[Interface]
ListenPort = 51821
PrivateKey = ***

[Peer]
PublicKey = ***
AllowedIPs = 10.101.0.2/32, 192.168.1.0/24
Endpoint = xxx.xxx.xxx.xxx:51825
PersistentKeepalive = 20

[Peer]
PublicKey = ***
AllowedIPs = 10.101.255.254/32
Endpoint = xxx.xxx.xxx.xxx:32765

[Peer]
PublicKey = ***
AllowedIPs = 10.101.255.252/32

[Peer]
PublicKey = ***
AllowedIPs = 10.101.255.253/32

However these settings are being overwritten by (i guess) netclient and nothing works after a couple of minutes.

I know it takes time to fix bugs, but is there a workaround I to keep these settings permanent, while this is being fixed? I am sure other people would find it useful

PS thanks for making such a great product!

I have the same problem and managed to work out what is not working, I hope this can help.

My network is like the one in the graph below

so my clients connect to VPS-1 (ingreess server) and i would like to access the 192.168.1.0/24 subnet on docker-01 (egress) This does not work.

After playing around with the wireguard settings I figured out it works (but only for a few minutes) if add the subnet to the WG allowed ips in both nodes (vps-1 and docker-01) examples below of wireguard config files I used

On docker-01
[Interface]
ListenPort = 51825
PrivateKey = ***

[Peer]
PublicKey = ***
AllowedIPs = 10.101.0.1/32, 0.0.0.0/0, 10.101.255.254/32, 10.101.255.252/32, 10.101.255.253/32, 192.168.1.0/24
Endpoint = 140.xxx.xxx.xxx:51821
PersistentKeepalive = 20
on vps-1
[Interface]
ListenPort = 51821
PrivateKey = ***

[Peer]
PublicKey = ***
AllowedIPs = 10.101.0.2/32, 192.168.1.0/24
Endpoint = xxx.xxx.xxx.xxx:51825
PersistentKeepalive = 20

[Peer]
PublicKey = ***
AllowedIPs = 10.101.255.254/32
Endpoint = xxx.xxx.xxx.xxx:32765

[Peer]
PublicKey = ***
AllowedIPs = 10.101.255.252/32

[Peer]
PublicKey = ***
AllowedIPs = 10.101.255.253/32
However these settings are being overwritten by (i guess) netclient and nothing works after a couple of minutes.

I know it takes time to fix bugs, but is there a workaround I to keep these settings permanent, while this is being fixed? I am sure other people would find it useful

PS thanks for making such a great product!

Can verify the issue and the "solution". To be more precise: the changes from a netclient pull are reflected in the /etc/netclient/netclient.yml file, but wireguard isn't configured accordingly.

The command wg set netmaker peer J5Vg....DjopilE= allowed-ips "10.129.137.1/32, 192.168.178.0/24" works but gets overwritten within seconds.

Hope this helps!

Best, Lukas

gravitl / netmaker