cargnellogiacomo commented 1 year ago

Hi,

I am currently using NordVPN Linux, installed on a Ubuntu Server, as a VPN gateway/router for the devices on my network that do not have a native NordVPN app. From version 3.16.5, the traffic is not anymore transferred through the NordVPN Linux. If I downgrade to version 3.16.3, it works again. Any suggestions? Thank you and regards

keliramu commented 1 year ago

Hi, thank you for feedback. We have some suspicions, but maybe you could gives us more information on how your setup looks like? e.g. your ubuntu server as gateway - does it have 2 network interfaces to separate internal subnet from internet, or you have one network interface and some sort of routing setup?

If possible, send us output of commands (on ubuntu server):

ip a
ip rule
ip route show table all

Best regards.

cargnellogiacomo commented 1 year ago

Here we go:

ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 02:11:32:2c:47:9a brd ff:ff:ff:ff:ff:ff altname enp0s3 inet 192.168.62.100/24 brd 192.168.62.255 scope global ens3 valid_lft forever preferred_lft forever inet6 fe80::11:32ff:fe2c:479a/64 scope link valid_lft forever preferred_lft forever 6: nordtun: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 500 link/none inet 10.8.2.4/24 scope global nordtun valid_lft forever preferred_lft forever

ip rule 0: from all lookup local 32764: from all lookup main suppress_prefixlength 0 32765: not from all fwmark 0xe1f1 lookup 205 32766: from all lookup main 32767: from all lookup default

ip route show table all default dev nordtun table 205 scope link default via 192.168.62.1 dev ens3 proto static 10.8.2.0/24 dev nordtun proto kernel scope link src 10.8.2.4 192.168.62.0/24 dev ens3 proto kernel scope link src 192.168.62.100 local 10.8.2.4 dev nordtun table local proto kernel scope host src 10.8.2.4 broadcast 10.8.2.255 dev nordtun table local proto kernel scope link src 10.8.2.4 local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1 local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1 broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1 local 192.168.62.100 dev ens3 table local proto kernel scope host src 192.168.62.100 broadcast 192.168.62.255 dev ens3 table local proto kernel scope link src 192.168.62.100 fe80::/64 dev ens3 proto kernel metric 256 pref medium local fe80::11:32ff:fe2c:479a dev ens3 table local proto kernel metric 0 pref medium multicast ff00::/8 dev ens3 table local proto kernel metric 256 pref medium

EmilijusS commented 1 year ago

Is this with version 3.16.3 or 3.16.5? And whichever version this was, could you also post the output while using the other version as well?

zearthlink commented 1 year ago

This issues happens to me also in version 3.16.5 (haven't tested .4). 3.16.3 works fine. All the outputs from iptables (nat) , ip route show table all etc were the same so couldn't really understand why it wasn't working.. after reverting to version .3, all good... Couldn't see much in the tcp dump :S

@vpn-gateway:~$ sudo tcpdump -vvv host 8.8.8.8 tcpdump: listening on ens18, link-type EN10MB (Ethernet), snapshot length 262144 bytes 12:12:22.728456 IP (tos 0x0, ttl 1, id 6570, offset 0, flags [DF], proto UDP (17), length 37) 192.168.8.175.37228 > dns.google.33434: [bad udp cksum 0xd989 -> 0xdb27!] UDP, length 9 12:12:22.728650 IP (tos 0x0, ttl 1, id 6571, offset 0, flags [DF], proto UDP (17), length 37) 192.168.8.175.37228 > dns.google.33434: [bad udp cksum 0xd989 -> 0xdb27!] UDP, length 9 12:12:22.728776 IP (tos 0x0, ttl 1, id 6572, offset 0, flags [DF], proto UDP (17), length 37) 192.168.8.175.37228 > dns.google.33434: [bad udp cksum 0xd989 -> 0xdb27!] UDP, length 9 12:12:22.728934 IP (tos 0x0, ttl 2, id 6573, offset 0, flags [DF], proto UDP (17), length 37) 192.168.8.175.37228 > dns.google.33435: [bad udp cksum 0xd989 -> 0xdb26!] UDP, length 9

edit: I have 2 VMs , 1 with the 16.3 and another with the 16.5 so if you want something other log, just write it down and I'll see what I can get

cargnellogiacomo commented 1 year ago

Is this with version 3.16.3 or 3.16.5? And whichever version this was, could you also post the output while using the other version as well?

This is the output from 3.16.3 and the output from 3.16.5 is the same, as @zearthlink indicated

EmilijusS commented 1 year ago

Indeed the provided outputs don't show anything suspicious. We will try to reproduce the issue and if we manage that then I'm sure we will find the root cause. Could you please provide exact details about your setup? What steps did you take to set up your Ubuntu server to act as a router? How do external devices connect to it?

zearthlink commented 1 year ago

In my case:

What I have is Proxmox installed in Dell OptiPlex 3090 Micro/i5-10500T/32GB DDR4/256GB SSD/External USB HDD

pveversion -v

proxmox-ve: 8.0.1 (running kernel: 6.2.16-3-pve) pve-manager: 8.0.3 (running version: 8.0.3/bbf3993334bfa916) pve-kernel-6.2: 8.0.2 pve-kernel-6.2.16-3-pve: 6.2.16-3 ceph-fuse: 17.2.6-pve1+3 corosync: 3.1.7-pve3 criu: 3.17.1-2 glusterfs-client: 10.3-5 ifupdown2: 3.2.0-1+pmx2 ksm-control-daemon: 1.4-1 libjs-extjs: 7.0.0-3 libknet1: 1.25-pve1 libproxmox-acme-perl: 1.4.6 libproxmox-backup-qemu0: 1.4.0 libproxmox-rs-perl: 0.3.0 libpve-access-control: 8.0.3 libpve-apiclient-perl: 3.3.1 libpve-common-perl: 8.0.5 libpve-guest-common-perl: 5.0.3 libpve-http-server-perl: 5.0.3 libpve-rs-perl: 0.8.3 libpve-storage-perl: 8.0.1 libspice-server1: 0.15.1-1 lvm2: 2.03.16-2 lxc-pve: 5.0.2-4 lxcfs: 5.0.3-pve3 novnc-pve: 1.4.0-2 proxmox-backup-client: 2.99.0-1 proxmox-backup-file-restore: 2.99.0-1 proxmox-kernel-helper: 8.0.2 proxmox-mail-forward: 0.1.1-1 proxmox-mini-journalreader: 1.4.0 proxmox-widget-toolkit: 4.0.5 pve-cluster: 8.0.1 pve-container: 5.0.3 pve-docs: 8.0.3 pve-edk2-firmware: 3.20230228-4 pve-firewall: 5.0.2 pve-firmware: 3.7-1 pve-ha-manager: 4.0.2 pve-i18n: 3.0.4 pve-qemu-kvm: 8.0.2-3 pve-xtermjs: 4.16.0-3 qemu-server: 8.0.6 smartmontools: 7.3-pve1 spiceterm: 3.3.0 swtpm: 0.8.0+pve1 vncterm: 1.8.0 zfsutils-linux: 2.1.12-pve1

Linux Server Version:

ubuntu-22.04.3-live-server-amd64

Conf file:

/etc/pve/qemu-server# cat 100.conf bios: seabios boot: order=scsi0 cores: 2 ide2: none,media=cdrom memory: 4096 meta: creation-qemu=7.2.0,ctime=1682528116 name: VPN-GATEWAY net0: virtio=2A:90:D0:99:39:EB,bridge=vmbr0,firewall=1 numa: 0 ostype: l26 scsi0: local-lvm:vm-100-disk-0,iothread=1,size=32G scsihw: virtio-scsi-single smbios1: uuid=39530b42-db29-4bee-be5f-a71840d11a77 sockets: 1 startup: order=1,up=1 vmgenid: 9e91ad3f-0cb5-45e8-9a7d-4adf327e958d

Topology

Nordvpn Linux Server (NLS) with IP address (192.168.8.253) will act as GW to all the VM's inside my Proxmox.
Default gateway for NLS is my physical router (192.168.8.1)
Subnet is 192.168.8.0/24
DNS is 8.8.8.8, 1.1.1.1

Nordvpn Settings

Technology: NORDLYNX Firewall: enabled Firewall Mark: 0xe1f1 Routing: enabled Analytics: enabled Kill Switch: enabled Threat Protection Lite: disabled Notify: disabled Auto-connect: enabled IPv6: disabled Meshnet: disabled DNS: disabled

frozen905 commented 1 year ago

WOW!

Same issue here!

I've been posting on ArchLinux forums about it all week, looking for a solution. Please see my post here for all relevant testing and output to date so far, rather than me spamming it here: https://bbs.archlinux.org/viewtopic.php?id=288588

I too can no longer use NordVPN as a system-wide gateway. I had to downgrade to 3.15.0 and it works absolutely flawlessly. On latest versions however, attempts to use the gateway are passing only unencrypted traffic to clients using it as a gateway. IP is showing up unsecured by the VPN. Not good.

Nord needs this fixed - these latest releases are 100% useless as far as I'm concerned, the app has not been working since 3.15.0 or whichever iteration it goes to before gateway functionality broke.

This needs to be a top priority, the app has a glaring security gap in it

Jauler commented 1 year ago

Hi,

Thank you for your feedback.

I tried reproducing this issue by using two docker containers (debian:12) while one was running NordVPN Linux application (v3.16.5) using iptables masquarade rules posted in ArchLinux forums:

iptables -t nat -A POSTROUTING -o nordlynx -j MASQUERADE
iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i ens18 -o nordlynx -j ACCEPT

and in the client machine, as I have no DHCP server in docker, I just changed the default gateway by using:

ip route delete default 
ip route add default via 172.17.0.7 dev eth0

And later just used curl to issue a HTTP request:

# curl -s ifconfig.co/json | jq ".ip, .asn_org"
"45.82.33.87"
"Packethub s.a."

The above setup worked as expected, and client container had network connectivity over Nord VPN servers. Therefore it seems like there is more to the issue than just the above settings. In order to help continue searching for root cause, would anyone with the reproduction be available to do a few things:

run ip rule show and ip route show table all on the gateway client (if the gateway client is android or linux) and share the output
run iptables-save on gateway and gateway client as well and share the output (if you are using iptables<->nftables translation layer, and some custom nftables rules -> then you can use nft list ruleset).
run for i in $(find /proc/sys -iname '*redirect*'); do echo $i; cat $i; done on both, GW and GW client, and share the output

I am hoping that the above information will be enough to figure out the issue, but in case it is not, traffic captures on the gateway as well as client would be very helpful.

Thanks in advance for your help!

Jauler commented 1 year ago

And of course, double-checking that /proc/sys/net/ip_forward is enabled would be healthy :)

cargnellogiacomo commented 1 year ago

Indeed the provided outputs don't show anything suspicious. We will try to reproduce the issue and if we manage that then I'm sure we will find the root cause. Could you please provide exact details about your setup? What steps did you take to set up your Ubuntu server to act as a router? How do external devices connect to it?

Here below my configuration. The devices on my network simply use as gateway the ip of the Ubuntu Server and the traffic is routed through the server.

Ubuntu Server:

Distributor ID: Ubuntu Description: Ubuntu 22.04.3 LTS Release: 22.04 Codename: jammy

IFCONFIG

ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.62.100 netmask 255.255.255.0 broadcast 192.168.62.255 inet6 fe80::11:32ff:fe2c:479a prefixlen 64 scopeid 0x20 ether 02:11:32:2c:47:9a txqueuelen 1000 (Ethernet) RX packets 284470092 bytes 281690771827 (281.6 GB) RX errors 0 dropped 15651 overruns 0 frame 0 TX packets 273147154 bytes 270138929851 (270.1 GB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 loop txqueuelen 1000 (Local Loopback) RX packets 5154 bytes 257700 (257.7 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 5154 bytes 257700 (257.7 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

nordtun: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST> mtu 1500 inet 10.8.3.10 netmask 255.255.255.0 destination 10.8.3.10 unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 500 (UNSPEC) RX packets 10068931 bytes 12357134711 (12.3 GB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 4675466 bytes 1795010821 (1.7 GB) TX errors 0 dropped 1225 overruns 0 carrier 0 collisions 0

IPTABLES -L

Chain INPUT (policy DROP) target prot opt source ACCEPT udp -- anywhere ACCEPT udp -- anywhere ACCEPT udp -- anywhere ACCEPT udp -- anywhere ACCEPT udp -- anywhere ACCEPT udp -- anywhere ACCEPT udp -- anywhere ACCEPT udp -- anywhere ACCEPT tcp -- anywhere ACCEPT tcp -- anywhere ACCEPT tcp -- anywhere ACCEPT tcp -- anywhere ACCEPT tcp -- anywhere ACCEPT tcp -- anywhere ACCEPT tcp -- anywhere ACCEPT tcp -- anywhere ACCEPT all -- 192.168.62.0/24 ACCEPT all -- anywhere DROP all -- anywhere ACCEPT all -- anywhere ACCEPT icmp -- 192.168.62.0/24 ACCEPT tcp -- 192.168.62.0/24 ACCEPT tcp -- 192.168.62.0/24 ACCEPT udp -- 192.168.62.0/24 ACCEPT udp -- 192.168.62.0/24 ACCEPT tcp -- 192.168.62.0/24 ACCEPT tcp -- 192.168.62.0/24 ACCEPT udp -- anywhere ACCEPT udp -- anywhere ACCEPT all -- anywhere destination
anywhere udp dpt:445 / nordvpn / anywhere udp spt:445 / nordvpn / anywhere udp dpts:netbios-ns:139 / nordvpn / anywhere udp spts:netbios-ns:139 / nordvpn / anywhere udp dpt:80 / nordvpn / anywhere udp spt:80 / nordvpn / anywhere udp dpt:22 / nordvpn / anywhere udp spt:22 / nordvpn / anywhere tcp dpt:microsoft-ds / nordvpn / anywhere tcp spt:microsoft-ds / nordvpn / anywhere tcp dpts:137:netbios-ssn / nordvpn / anywhere tcp spts:137:netbios-ssn / nordvpn / anywhere tcp dpt:http / nordvpn / anywhere tcp spt:http / nordvpn / anywhere tcp dpt:ssh / nordvpn / anywhere tcp spt:ssh / nordvpn / anywhere / nordvpn / anywhere connmark match 0xe1f1 / nordvpn / anywhere / nordvpn / anywhere
anywhere
anywhere tcp dpt:ssh anywhere state NEW tcp dpt:http anywhere state NEW udp dpt:netbios-ns anywhere state NEW udp dpt:netbios-dgm anywhere state NEW tcp dpt:netbios-ssn anywhere state NEW tcp dpt:microsoft-ds anywhere state NEW udp dpt:51413 anywhere state NEW udp dpt:51413 anywhere state RELATED,ESTABLISHED

Chain FORWARD (policy DROP) target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED ACCEPT all -- anywhere anywhere

Chain OUTPUT (policy ACCEPT) target prot opt source destination
ACCEPT udp -- anywhere anywhere udp dpt:445 / nordvpn / ACCEPT udp -- anywhere anywhere udp spt:445 / nordvpn / ACCEPT udp -- anywhere anywhere udp dpts:netbios-ns:139 / nordvpn / ACCEPT udp -- anywhere anywhere udp spts:netbios-ns:139 / nordvpn / ACCEPT udp -- anywhere anywhere udp dpt:80 / nordvpn / ACCEPT udp -- anywhere anywhere udp spt:80 / nordvpn / ACCEPT udp -- anywhere anywhere udp dpt:22 / nordvpn / ACCEPT udp -- anywhere anywhere udp spt:22 / nordvpn / ACCEPT tcp -- anywhere anywhere tcp dpt:microsoft-ds / nordvpn / ACCEPT tcp -- anywhere anywhere tcp spt:microsoft-ds / nordvpn / ACCEPT tcp -- anywhere anywhere tcp dpts:137:netbios-ssn / nordvpn / ACCEPT tcp -- anywhere anywhere tcp spts:137:netbios-ssn / nordvpn / ACCEPT tcp -- anywhere anywhere tcp dpt:http / nordvpn / ACCEPT tcp -- anywhere anywhere tcp spt:http / nordvpn / ACCEPT tcp -- anywhere anywhere tcp dpt:ssh / nordvpn / ACCEPT tcp -- anywhere anywhere tcp spt:ssh / nordvpn / ACCEPT all -- anywhere 192.168.62.0/24 / nordvpn / CONNMARK all -- anywhere anywhere mark match 0xe1f1 / nordvpn / CONNMARK save ACCEPT all -- anywhere anywhere connmark match 0xe1f1 / nordvpn / DROP all -- anywhere anywhere / nordvpn /

NordVPN settings:

Technology: OPENVPN Protocol: UDP Firewall: enabled Firewall Mark: 0xe1f1 Routing: enabled Analytics: disabled Kill Switch: enabled Threat Protection Lite: disabled Obfuscate: disabled Notify: enabled Auto-connect: enabled IPv6: disabled Meshnet: disabled DNS: 192.168.62.2 Whitelisted ports: 22 (UDP|TCP) 80 (UDP|TCP) 137 - 139 (UDP|TCP) 445 (UDP|TCP) Whitelisted subnets: 192.168.62.0/24

EmilijusS commented 1 year ago

@cargnellogiacomo In your case the problem might be the whitelisting. Since 3.16.4 the app is inserting additional firewall rules which ensure that traffic from whitelisted subnets/ports will never pass through the VPN network adapter (https://github.com/NordSecurity/nordvpn-linux/issues/7). So in your case if you want to provide VPN to devices forwarding through the gateway, the IPs of those devices must not be whitelisted. Also ideally you should try not using port whitelisting in such case because unfortunately there's no way to specify subnets for the ports being whitelisted, so all traffic on those ports will also never go through VPN.

Other users here seem to have the same issue without using whitelisting, so we are still waiting for more info about that case as written here https://github.com/NordSecurity/nordvpn-linux/issues/50#issuecomment-1715750516

cargnellogiacomo commented 1 year ago

@cargnellogiacomo In your case the problem might be the whitelisting. Since 3.16.4 the app is inserting additional firewall rules which ensure that traffic from whitelisted subnets/ports will never pass through the VPN network adapter (#7). So in your case if you want to provide VPN to devices forwarding through the gateway, the IPs of those devices must not be whitelisted. Also ideally you should try not using port whitelisting in such case because unfortunately there's no way to specify subnets for the ports being whitelisted, so all traffic on those ports will also never go through VPN.

Other users here seem to have the same issue without using whitelisting, so we are still waiting for more info about that case as written here #50 (comment)

Hi,

I can confirm that I was able to use the new nordvpn version, removing the subnet from the whitelist and adding some other specific port in the whitelist to make sure that some services can be reached by the server. it works fine! thank you a lot.

frozen905 commented 1 year ago

edit - I'm going to re-do all of this, I've confused myself and it's a lot to read

NordSecurity / nordvpn-linux

Traffic blocked if used as VPN gateway/router from version 3.16.5 #50

pveversion -v

Linux Server Version:

Conf file:

Topology

Nordvpn Settings