Is it possible to use a commercial VPN in combination with the multipath WAN connections?

[xdaviddx]

I've recently set up OpenMPTCProuter and most things are working fine. The one problem I have is that Netflix, Hulu, etc. block the VPS server's IP address. I know I can try to exclude the streaming services from going through the OpenMPTCProuter, but I'd like to have the extra bandwidth for them, and it also isn't easy to get them to be excluded. I could only get it working with the domain name option, and finding all possible domain names is difficult.

What I want to do is to go from my LAN to the VPS server, and then have the VPS server run the VPN client to a commercial VPN server (expressvpn is what I'm using currently). I did install the VPN client on the VPS server and got it to run. I believe I verified, at the command line of the VPS server, that the tunnel was active. I forget how I did that. Maybe by looking at ping times with and without it active. I don't remember exactly. But the problem was that OpenMPTCProuter traffic wasn't going through that tunnel. I'm not exactly sure how to set up routing in the VPS so that the two are linked together like that.

As a temporary alternative, I tried some other things:

1) I ran the vpn client on my PC, with that vpn tunnel going through OpenMPTCProuter just fine. I'm guessing there are some efficiency losses here, because there is encryption on top of encryption between the OpenMPTCProuter router and the VPS software. This isn't a solution that will work for my network, since I can't run the client on all devices.

2) I ran the openvpn client on a router running Tomato. That router sits between my LAN devices and the OpenMPTCProuter. It does work, but the performance is really slow. Looking at the cpu usage on that router, I think I'm hitting a cpu bottleneck. It is an older router, so resource limited.

3) I tried to configure the openvpn client in the OpenMPTCProuter, thinking that it would behave like number (2) above, but would be running in the same machine as the multi-path code. I was able to get the tunnel up, but I can't get any traffic to go across it. I've tried various settings for creating an interface and firewall zone in the UI, but nothing seems to work. I'm starting to wonder if this openvpn option is strictly for use by OpenMPTCProuter to use in place of other tunneling options between the router and the VPS server, not in the way I want to use it?

Do you have any ideas on how to make (1) work and/or how to make (3) work, if it is possible? I think I'm close to being able to use OpenMPTCProuter, but I need to solve the commercial VPN issue.

Thanks.

[Ysurac]

VPN can be on VPS side, but you have to configure route & firewall via shorewall. OpenVPN can be used on the router but you have to configure routes and firewall for that.

[xdaviddx]

VPN can be on VPS side, but you have to configure route & firewall via shorewall.

Thank you for verifying that it can be done and for pointing me toward shorewall to make the configuration easier. My problem if I put the VPN client on the VPS is that I won't be able to do a VPN bypass based on source address.

However, a person could use two VPS systems and two OMR routers if they wanted to do things this way. They would route packets based on source address to one OMR router (that points to a VPS without a VPN client running on it) or another OMR router (that points to a VPS with a VPN client running on it). The trade off here is the extra cost of a router for OMR and the extra cost per month of the VPS system.

It would look like this:

LAN device A --> OMR A VM --> VPS X (with OMR server running on port A) --> VPN client --> VPN server --> Internet LAN device B --> OMR B VM --> VPS X (with OMR server running on port B) --> Internet

IF two copies of OMR server can be run on the VPS server at the same time, on different ports, then there wouldn't be the cost of two VPS servers. I don't know if that is possible.

IF someone runs OMR router as virtual machines on the client side, then there isn't a cost of another piece of router hardware either.

Therefore, if a person ran two copies of OMR server (if possible) on the same VPS server and if a person ran OMR router on two virtual machines on the client side, then there isn't an additional cost (other than the slice of the VM host that provides the VM for the OMR router client firmware).

Is there anything preventing two copies of OMR server from running on the same VPS instance?

OpenVPN can be used on the router but you have to configure routes and firewall for that.

I need more experience with openwrt routes and firewalls. It is something I'll need to learn more about. Thanks for verifying it can be done. Running shorewall with webmin on the OMR router may help me with this too, if it is easier to configure than the openwrt firewall and routes.

I think I may run the VPN client on a separate router (or in a VM), to make things a little simpler, using vlans for routing. That's if I have the VPN client at home. If I do the above, where the VPN client runs on the VPS, then that may be more efficient, since it won't be a tunnel inside of a tunnel.

The other reason someone may want to run the VPN client on the OMR VPS server is because it will have AES instructions in the CPU, which can reduce the risk of the VPN network throughput being reduced due to CPU overload. If the VPN client is run on a router, there is a chance it won't have AES instructions in the CPU and won't have a fast enough CPU, so the network throughput will be slowed down. This is especially true if they are trying to use lower cost hardware, such as Raspberry Pi, where AES instructions are not implemented in the CPU.

Thanks again!

[lonacluka]

@xdaviddx Hi! Im actually trying to do the same thing that you did.

1) When I did option number 1, I noticed VPN traffic goes only through 1 of the WAN interfaces (no multipathing/bonding). 2) Same thing. Traffic going through WAN link that is marked as "master" in OMR config 3) Succeeded doing this by creating new tun1 interface. Importing nordvpn ovpn config in GUI and it started. Maybe your VPN doesnt use 443 port like nordvpn so you have to set up port forwarding to open port. I used OMR-bypass tool to force my TV's source ip address going to tun1(nordvpn/openvpn) and it works, I have usa netflix/hulu but still traffic going only through 1 WAN interface, so no bandwidth benefit :/

I think your idea is actually great, to configure second vps but will not be easy for me since I have limited networking exp and zero iptables/shorewall knowledge. Did you manage to figure out something? If you did if you just share a few pointers (or a few commands that you used, that I can use as a bluprint) I would be grateful. Thanks!

[xdaviddx]

@luka-diplomski Hi Luka,

It seems like something must be off for (1) and (2) to not use both WAN links. I can't remember anymore if I looked to verify that when I did my testing. I don't have my network configured currently to be able to test OMR with 2 WAN links. I'm waiting for another issue to be fixed where it isn't working properly with VLANs if one of the WAN links goes down and comes back up. Does OpenMPTCProuter do dynamic packet balancing for each TCP connection?

Next time I have it set up for testing both links, I can look at whether it uses both links when the commercial VPN is being used.

I didn't try any other testing with running the VPN client on the OMR client side or the VPS side. Instead, I bought a used machine to use as a VM server and I'm running a virtualized instance OpenWRT + OpenVPN to do the commercial VPN (with OpenVPN as the client) and another VM where I am running OMR. OMR sits on the WAN side of the first virtualized router I mentioned.

This won't be as efficient as running the VPN client on the VPS server. I may try that approach at some future point in time, after I get the current approach working.

The current approach is suffering from this problem: VPN tunnel down and loss conection when speed test Most likely the Glorytun TCP software doesn't deal well with bufferbloat. It gives up when the latency gets too great, bringing the tunnel down and causing lost TCP connections in the process. This doesn't work for what I need this for. I've been testing currently with just a single WAN connection. I'm hoping when I test with 2 WAN connections again, and have more bandwidth available, this won't happen as often. I can make it happen currently when streaming one or more movies that requires more bandwidth than my WAN connection has at the time, or by doing speed tests.

The current approach's issue probably isn't directly related to what I am doing with the commercial VPN. It is probably just that when using the commercial VPN, I have less bandwidth available (because it is a less efficient use of the connection when you have a VPN on top of a VPN), so that makes the bufferbloat happen sooner.

All this is to say that I haven't spent time trying to get the commercial VPN client running on the VPS server. As I mentioned, this approach would seem to be the most efficient use of bandwidth, but it means everything has to flow through the VPN -- unless someone has 2 OMR/VPS systems, one with VPN and one without.

What is your goal for your system?

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days

[grapexy]

Spent some time fiddling with shorewall, so for any lonely souls trying to configure VPN on the VPS side, I'll make a quick guide for you. We'll be doing Wireguard as openvpn will absolutely suffocate any budget VPS under load (and budget VPS is really all you need).

1. Allow UDP connections to WG port 51820 (adjust if different port is used by provider). And optionally allow port 10000 access for webmin.

# /etc/shorewall/rules. After ?SECTION NEW

# Allow external wireguard
ACCEPT      $FW     net     udp 51820

# [Optional] Allow webmin
ACCEPT      net     $FW     tcp 10000

2. Restart shorewall:

   shorewall restart

3. [Optional] Install webmin if you prefer doing things in GUI (it's best if you uninstall it after you're done and remove port 10000 from above):

   wget http://prdownloads.sourceforge.net/webadmin/webmin_1.994_all.deb && dpkg -i webmin_1.994_all.deb

4. Disable IPv6. (otherwise you'll have to configure firewall rules separately for ipv6 and if you can handle ipv6 complexities, you probably don't need this guide).

# /etc/shorewall/shorewall.conf
# ...

# DISABLE_IPV6=No
DISABLE_IPV6=Yes

# ...

6. Configure killswitch and allow vpn-expvn traffic. This ensures that all traffic only leaves VPS via VPN:

# /etc/shorewall/policy

###############################################################################
#SOURCE         DEST            POLICY          LOG LEVEL       LIMIT:BURST
vpn             exvpn           ACCEPT                          # [custom] Allow vpn to external VPN
vpn             net             DROP            warning         # [custom] Drop VPN to direct net
vpn             fw              ACCEPT
fw              exvpn           ACCEPT                          # [custom] Allow FW to external VPN
fw              vpn             ACCEPT
fw              net             DROP            warning
net             all             DROP            debug
vpn             vpn             DROP            debug
# THE FOLLOWING POLICY MUST BE LAST
all             all             REJECT          debug

7. Add VPN interface and add it to exvpn zone:

# /etc/shorewall/interfaces

# ...

exvpn exwg+ nosmurfs,tcpflags

8. Create /etc/wireguard/exwg0.conf and fill in necessary information:

   
   [Interface]
   PrivateKey = {FROM_VPN_PROVIDER}
   Address = {FROM_VPN_PROVIDER}
   DNS = 1.1.1.1

Use server IP for replying back to direct connections instead of VPN

PostUp = ip rule add from {SERVER_PUBLIC_IP} table main

NAT rules

PostUp = iptables -t nat -A POSTROUTING -o exwg0 -j MASQUERADE

PostDown = ip rule del from {SERVER_PUBLIC_IP} table main

PostDown = iptables -t nat -D POSTROUTING -o exwg0 -j MASQUERADE;

[Peer] PublicKey = {FROM_VPN_PROVIDER} AllowedIPs = 0.0.0.0/0 Endpoint = {FROM_VPN_PROVIDER}:51820

9. Change `$NET_IFACE` to `exwg0` in NAT rules:

/etc/shorewall/snat

ACTION SOURCE DEST PROTO PORT IPSEC MARK USER SWITCH ORIGDEST PROBABILITY

# MASQUERADE 10.255.247.0/24,\ 10.255.248.0/24,\ 10.255.250.0/24,\ 10.255.251.0/24,\ 10.255.252.0/24,\ 10.255.253.0/24,\ 10.255.254.0/24,\ 10.255.255.0/24,\ 169.254.0.0/16,\ 172.16.0.0/12,\ 192.168.0.0/16 $NET_IFACE

10. Restart shorewall:

shorewall restart

11. We haven't started VPN yet, so verify that killswitch is working. Run `curl ifconfig.me` from VPS and from OMR / OMR client. Requests should time out. If you got your VPS IP, you did something wrong.
12. Enable wireguard config autostart: `systemctl enable --now wg-quick@exwg0`
13. Restart machine just in case `/sbin/reboot` and run `curl ifconfig.me`. You should get VPN IP from both VPS and OMR / OMR clients.

For port forwarding, follow the port forwarding guide from wiki. OMR will create DNAT rule on VPS (`/etc/shorewall/rules`) if "use v2ray" is not checked in port forwarding settings (i.e. use vpn). If "use v2ray" is checked, it will create ACCEPT rule (as v2ray is listening directly on firewall for that port):

/etc/shorewall/rules

Auto created rule for vpn

DNAT net vpn:$OMR_ADDR udp 54348 # OMR openmptcprouter redirect router 54348 port tcp

Auto created rule for v2ray

ACCEPT net $FW udp 54348 # OMR openmptcprouter open router 54348 port udp

This obviously won't work, as source is not going to be `net`, but `exvpn` so we'll need to duplicate the above, but replace the source:

/etc/shorewall/rules

DNAT exvpn vpn:$OMR_ADDR udp 54348 ACCEPT exvpn $FW udp 54348

Just a side note, I noticed that v2ray works best in this configuration, but I'm running OMR on a powerful Mini PC, so YMMV. Check `htop` on both VPS and OMR under load with different configurations and you'll see what's best.

@Ysurac My knowledge in this area is couple of hours old. Could you please verify that everything looks correct?

[hle5128]

Spent some time fiddling with shorewall, so for any lonely souls trying to configure VPN on the VPS side, I'll make a quick guide for you. We'll be doing Wireguard as openvpn will absolutely suffocate any budget VPS under load (and budget VPS is really all you need).

1. Allow UDP connections to WG port 51820 (adjust if different port is used by provider). And optionally allow port 10000 access for webmin.

# /etc/shorewall/rules. After ?SECTION NEW

# Allow external wireguard
ACCEPT        $FW     net     udp 51820

# [Optional] Allow webmin
ACCEPT        net     $FW     tcp 10000

2. Restart shorewall:

shorewall restart

3. [Optional] Install webmin if you prefer doing things in GUI (it's best if you uninstall it after you're done and remove port 10000 from above):

wget http://prdownloads.sourceforge.net/webadmin/webmin_1.994_all.deb && dpkg -i webmin_1.994_all.deb

5. Disable IPv6. (otherwise you'll have to configure firewall rules separately for ipv6 and if you can handle ipv6 complexities, you probably don't need this guide).

# /etc/shorewall/shorewall.conf
# ...

# DISABLE_IPV6=No
DISABLE_IPV6=Yes

# ...

6. Configure killswitch and allow vpn-expvn traffic. This ensures that all traffic only leaves VPS via VPN:

# /etc/shorewall/policy

###############################################################################
#SOURCE         DEST            POLICY          LOG LEVEL       LIMIT:BURST
vpn             exvpn           ACCEPT                          # [custom] Allow vpn to external VPN
vpn             net             DROP            warning         # [custom] Drop VPN to direct net
vpn             fw              ACCEPT
fw              exvpn           ACCEPT                          # [custom] Allow FW to external VPN
fw              vpn             ACCEPT
fw              net             DROP            warning
net             all             DROP            debug
vpn             vpn             DROP            debug
# THE FOLLOWING POLICY MUST BE LAST
all             all             REJECT          debug

7. Add VPN interface and add it to exvpn zone:

# /etc/shorewall/interfaces

# ...

exvpn exwg+ nosmurfs,tcpflags

8. Create /etc/wireguard/exwg0.conf and fill in necessary information:

[Interface]
PrivateKey = {FROM_VPN_PROVIDER}
Address = {FROM_VPN_PROVIDER}
DNS = 1.1.1.1

# Use server IP for replying back to direct connections instead of VPN
PostUp = ip rule add from {SERVER_PUBLIC_IP} table main
# NAT rules
# PostUp = iptables -t nat -A POSTROUTING -o exwg0 -j MASQUERADE

PostDown = ip rule del from {SERVER_PUBLIC_IP} table main
# PostDown = iptables -t nat -D POSTROUTING -o exwg0 -j MASQUERADE;

[Peer]
PublicKey = {FROM_VPN_PROVIDER}
AllowedIPs = 0.0.0.0/0
Endpoint = {FROM_VPN_PROVIDER}:51820

9. Change $NET_IFACE to exwg0 in NAT rules:

# /etc/shorewall/snat

#ACTION                 SOURCE                  DEST            PROTO   PORT    IPSEC   MARK    USER    SWITCH  ORIGDEST        PROBABILITY
#
MASQUERADE              10.255.247.0/24,\
                        10.255.248.0/24,\
                        10.255.250.0/24,\
                        10.255.251.0/24,\
                        10.255.252.0/24,\
                        10.255.253.0/24,\
                        10.255.254.0/24,\
                        10.255.255.0/24,\
                        169.254.0.0/16,\
                        172.16.0.0/12,\
                        192.168.0.0/16          $NET_IFACE

10. Restart shorewall:

shorewall restart

11. We haven't started VPN yet, so verify that killswitch is working. Run curl ifconfig.me from VPS and from OMR / OMR client. Requests should time out. If you got your VPS IP, you did something wrong.
    12. Enable wireguard config autostart: systemctl enable --now wg-quick@exwg0
    13. Restart machine just in case /sbin/reboot and run curl ifconfig.me. You should get VPN IP from both VPS and OMR / OMR clients.

For port forwarding, follow the port forwarding guide from wiki. OMR will create DNAT rule on VPS (/etc/shorewall/rules) if "use v2ray" is not checked in port forwarding settings (i.e. use vpn). If "use v2ray" is checked, it will create ACCEPT rule (as v2ray is listening directly on firewall for that port):

#  /etc/shorewall/rules

# Auto created rule for vpn
DNAT            net             vpn:$OMR_ADDR   udp     54348   # OMR openmptcprouter redirect router 54348 port tcp
# Auto created rule for v2ray
ACCEPT        net     $FW     udp 54348   # OMR openmptcprouter open router 54348 port udp

This obviously won't work, as source is not going to be net, but exvpn so we'll need to duplicate the above, but replace the source:

#  /etc/shorewall/rules

DNAT            exvpn             vpn:$OMR_ADDR   udp     54348 
ACCEPT        exvpn       $FW     udp 54348   

Just a side note, I noticed that v2ray works best in this configuration, but I'm running OMR on a powerful Mini PC, so YMMV. Check htop on both VPS and OMR under load with different configurations and you'll see what's best.

@Ysurac My knowledge in this area is couple of hours old. Could you please verify that everything looks correct?

what is your speed like with running wireguard client on VPS side? If I understand correctly, wireguard IP is the public one and vps is just for mptcp. i assume you will lose quite bit of speed and latency?

[grapexy]

what is your speed like with running wireguard client on VPS side? If I understand correctly, wireguard IP is the public one and vps is just for mptcp. i assume you will lose quite bit of speed and latency?

Yes, commercial WG IP becomes the public IP.

I'm getting the same speed and latency as I was getting without commercial VPN on VPS. Important bit here is latency between the VPS and external VPN server, which in my case is in the same country as VPS and is is around 1ms, so it would not add up to anything meaningful. VPS has 1 Gigabit uplink and my VPN provider (Mullvad) has same, so it works out quite well.

[ioogithub]

@grapexy did you validate your guide, does it work? I have read your guide over a few times and I am going to give it a try. I noticed when searching though the issues that quite a few other users have attempted to do the same thing but met with limited success and another wireguard tutorial was attempted but didn't look like the user was successful.

but I'm running OMR on a powerful Mini PC

Curious to know what processor and mini PC you are using. I am currently researching what hardware I need for OMR.

@Ysurac does this look good to you, is anything missing? Perhaps we can move it to the tutorial section of the wiki. Currently, it is difficult to find because the title doesn't mention wireguard in the title so a user searching for a guide on how to use wireguard with OMR might miss it.

[ioogithub]

@grapexy I think I understand everything in your tutorial except a few lines in the /etc/wireguard/exwg0.conf: You have:

# Use server IP for replying back to direct connections instead of VPN
PostUp = ip rule add from {SERVER_PUBLIC_IP} table main
# NAT rules
# PostUp = iptables -t nat -A POSTROUTING -o exwg0 -j MASQUERADE

Should the #PostUp = iptables -t nat... line be commented out? Isn't this the actual line that does the work or are you referencing something existing that needs to be removed?

Same with the #PostDown line:

PostDown = ip rule del from {SERVER_PUBLIC_IP} table main
# PostDown = iptables -t nat -D POSTROUTING -o exwg0 -j MASQUERADE;

Could you explain what is going on here?

I generated a wireguard.conf file from a VPN provider as a comparison and although I don't completely understand what it is doing their lines are not commented out:

PostUp = iptables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT && ip6tables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT
PreDown = iptables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT && ip6tables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT

other than this point everything else seems understandable. Thank you for taking the time to post this tutorial.

[ioogithub]

@grapexy

This tutorial fails as step 10 Restart shorewall:

ERROR: Unknown zone (exvpn) /etc/shorewall/interfaces (line 27)

not sure what to do next, I added the interface in step 7 but it doesn't recognize the zone. I think there is a step missing here somewhere.

I would guess we need to add something to the /etc/shorewall/zones files, was this step omitted? What should the type of zone be ipv4?

Any ideas?

[ioogithub]

So I have tried this guide twice, and each time step 12 breaks the tunnel and you lose access to your VPS. If anyone in the future is reading this I would urge caution. I do not believe that this tutorial as it stands now works. There are missing steps and once wireguard is started it takes over and breaks the OMR tunnel.

Be careful.

[ioogithub]

Hey @grapexy,

I just tried your tutorial again. I get the same result as before, as soon as I get to step 12: Enable wireguard config autostart: systemctl enable --now wg-quick@exwg0 boom my ssh connection to the vps is cut and I have lost access to my vps. I guess its still running but there is no access.

I know for sure there is a missing section, You need to omitted creating the shorewall zone like this:

# /etc/shorewall/zones
# Zone    Type
...
exvpn      ipv4

It should be before #7 because you reference the zone in #7 but it doesn't exist yet. Any ideas on what else is missing or where the misconfiguration is?

This sucks because there is no way to recover from this. I have to destroy the VPS server and start again. Would it be possible to review your tutorial and finish what you started? It looks like it is almost complete and would be a helpful guide so trying to set this up.

[jhenkens]

@ioogithub I followed the OP's changes, with the addition of the shorewall zone, and used this as my policy.

#SOURCE         DEST            POLICY          LOG LEVEL       LIMIT:BURST

vpn             exvpn           ACCEPT # [custom] Allow VPN -> ExVPN
vpn             net             ACCEPT
vpn             fw              ACCEPT
fw              exvpn           ACCEPT # [custom] Allow EXVpn to talk externally
fw              vpn             ACCEPT
fw              net             ACCEPT
net             all             DROP
vpn     vpn     DROP
vpncl       vpn     ACCEPT
vpn     vpncl       ACCEPT
# THE FOLLOWING POLICY MUST BE LAST
all     all     REJECT

I needed to use the PostUp/PostDown rules (PostUp = ip rule add from 1.2.3.4 table main) or else my VPS was unreachable.

First time I tried, I made my VPS unusable and had to start from scratch.

I migrated over from Speedify+Smoothwan. I was getting 40/10 from Smoothwan, and am getting 120/15 from OMR. Both running in Proxmox on the same machine with ADSL+LTE, with a Wireguard server in a resi fiber to get around streaming restrictions.

[mrkaffeine92]

This should be implemented into the web GUI some way - the addition of a Wireguard public VPN client peering upstream from host VPS -> VPN.