search

zehome / MLVPN

Multi-link VPN (ADSL/SDSL/xDSL/Network aggregation / bonding)
http://www.mlvpn.fr/
BSD 2-Clause "Simplified" License
518 stars 127 forks source link

First attempts to use MLVPN (and shorewall) #68

Closed markfoodyburton closed 8 years ago

markfoodyburton commented 8 years ago

I’m also in France (in the countryside, with poor ADSL)… To make my life bearable, I use 2 ADSL links…

On my home router (a small Linux box), I run shorewall, and run the two ppp daemon’s. Hence I have 3 interfaces ppp0, ppp1 and eth0.

As an initial test, I installed mlvpn on both the server and the router (direct from the GIT in both cases).

I followed the instructions http://mlvpn.readthedocs.org/en/latest/linux_example.html - except I ONLY set up one VPN at this point - My intention is JUST to test the connection, not to attempt any aggregation yet! So, I took your example config and scripts, All I did was 1/ set mode = client/server on the client/server :-) 2/ set the client ip4 address to 10.42.42.2. Set the server ip4 address to 10.42.42.1 (/30 in both cases) 3/ set the mtg to 1432 (should this be the default?) 4/ set the password 5/ On the client, set an adsl connection binding the host to the address of one of the PPP endpoints - which should force things down one adsl path... [Side note, if you use 0.0.0.0 as the bindhost, you get multiple connections being made to the server through the 2 different paths, which is fun to watch] set port, and remote host to the server 6/ On the server side, I set the bind host to 0.0.0.0. In how ‘HowTo’ it does not mention a bindhost. If I do not set it, I get: dsl1 getaddrinfo(,5091) failed: Unknown error

Thats it I then start the server and the client. I’m happy to see: 2016-04-05T13:06:17 tunnel [dsl1] is up 2016-04-05T13:06:17 mlvpn0 up

Good. And I see plenty of keep alive packets going though if I do a -Dprotocol

So now I expect, on the client side, to be able to ‘ping’ the server. When I check the route, I do indeed have a 10.42.42. route: 10.42.42.0 0.0.0.0 255.255.255.252 U 0 0 0 mlvpn0

(and likewise on the server side) I can ping myself - e.g. on the client I can ping 10.42.42.2 successfully. But I can not ping the other side

ping 10.42.42.1

Fails….. :-(

Where am I going wrong? (I thought this was the easy bit, I’ve still got to get the integration with shore wall working :-))) )

Cheers Mark.

markfoodyburton commented 8 years ago

Ahh, yes sorry - I forgot to mention that

I set bindhost = “0.0.0.0" If I dont set it, then I get an error message, setting it to 0.0.0.0 or the external IP address all have the same result (127.0.0.1 only exposes the VPN server to the local server, so that won't work of course), the VPN comes up, but I’m unable to ‘ping’

Cheers

Mark.

on server under dsl1 did you setup "bindhost"?

—

markfoodyburton commented 8 years ago

To clarify more, on the client side, I set bindhost = 'x.x.x.x' - the external address of one of the PPP links. on the server side bindhost = '0.0.0.0' (I've tried with lots of different things, on the client and server side, but that doesn't seem to change anything, I must be missing something!)

zehome commented 8 years ago

maybe you have firewalled the tunnel interface?

Try

iptables -I INPUT -i mlvpn+ -j ACCEPT
iptables -I OUTPUT -o mlvpn+ -j ACCEPT
iptables -I FORWARD -i mlvpn+ -j ACCEPT
iptables -I FORWARD -o mlvpn+ -j ACCEPT

on both sides.

This will effectively disable your firewall with regard to mlvpn interfaces.

markfoodyburton commented 8 years ago

You are spot on - THANKYOU

paketecuento commented 8 years ago

I have done all those step, but cannot go any further than this

2016-06-25T18:37:34 [ DBG] absolute maximum mtu: 1444
2016-06-25T18:37:34 [INFO/config] new password set
2016-06-25T18:37:34 [INFO/config] reorder_buffer_size changed from 0 to 64
2016-06-25T18:37:34 [INFO/config] adsl1 tunnel added
2016-06-25T18:37:34 [INFO/config] adsl2 tunnel added
2016-06-25T18:37:34 [INFO] created interface `mlvpn0'
2016-06-25T18:37:34 [INFO] adsl1 bind to 0.0.0.0
2016-06-25T18:37:34 [INFO] adsl2 bind to 0.0.0.0

but no way to see the tunnel up :(

I'm running Debian Wheezzy on server and Jessie on client

markfoodyburton commented 8 years ago

@paketecuento hi, if your still having problems, let me know, I've got a nice working solution.

legolas108 commented 5 years ago

Having similar problems as @paketecuento, do you still offer your working solution, @markfoodyburton? Thanks much in advance!

markfoodyburton commented 5 years ago

I have to say, in the end, I left shorewall, and am now using firehol - one reason was because something in the shorewall setup ended up not playing nicely with mlvpn. I'd suggest moving across to firehol - the set up is fairly simple then

legolas108 commented 5 years ago

Thanks for quick reply, highly appreciated! Had a quick look at FireHOL. Would surely cover our firewall needs nicely. Just cannot see how it would help getting over the MLVPN connection issue. Would you share your MLVPN config?

markfoodyburton commented 5 years ago

I use macvlan's to set up my individual paths to the various providers as my 'modems' are all connected to the same switch. Hence I have (in /etc/network/interfaces)

auto macvlan5
iface macvlan5 inet static
    address 192.168.5.1
    netmask 255.255.255.0
    gateway 192.168.5.5 
       pre-up ip link add link eth0 name macvlan5 type macvlan mode vepa

Then in /etc/firehol/firehol.conf I snat snat4 to 192.168.5.1 outface macvlan5 src 192.168.1.0/24 dst not 192.168.0.0/16 I mark traffic on those links: connmark 0x5 interface macvlan5

Then set up your interfaces and routes as per normal.

Then you may also want to set up link-balancer I set up all the providers as gateways, and allow them to be a 'fallback' if mlvpn fails. Hence for an ADSL and an LTE, I have in my link-balancer.conf:

gateway ADSL5 dev macvlan5 gw 192.168.5.5
gateway LTE4 dev macvlan4 gw 192.168.4.4
gateway MLVPN dev mlvpn0 gw 10.42.42.1

table 50
    default via ADSL5

table 40
    default via LTE4

table 10
    default via MLVPN

# this is the main system routing table
table main
    default via MLVPN weight 250
    default via LTE4 weight 0
    default via ADSL5 weight 0

policy
        connmark 0x4 table 40
        connmark 0x5 table 50
        connmark 0x1 table 10
        rules mark 0x50 table 50
    rules src gw-src-ips LTE4 table 40
        rules src gw-src-ips ADSL5 table 50
        rules src gw-src-ips MLVPN table 10

I also make sure the link-balancer restarts if MLVPN goes up/down.... (do that in your mlvpn_updown.sh)

legolas108 commented 5 years ago

Thanks so much for taking the time! Will give it a try.

legolas108 commented 5 years ago

Wondering how you got the two ppp connections to run with both modems on the same switch with macvlans. Using in /etc/network/interfaces:

auto mv1
iface mv1 inet static
  address 192.168.10.2
  netmask 255.255.255.0
  gateway 192.168.10.10
  pre-up ip link add link eno1 name mv1 type macvlan mode vepa

auto dsl1
iface dsl1 inet ppp
  provider dsl1

and in /etc/ppp/peers/dsl1:

plugin rp-pppoe.so mv1
unit 0
user "********"
noauth
hide-password
persist
mtu 1492
noipdefault

works fine. Together with FireHOL firewall stable and secure Internet connection, really happy you pointed me to that!

And the 2nd connection just with different numbers and names works fine, too, if started by itself. But starting the 2nd connection when the first is running leads to immediate disconnect by remote peer:

May 25 21:25:05 dbsvr pppd[5413]: Plugin rp-pppoe.so loaded.
May 25 21:25:05 dbsvr pppd[5414]: pppd 2.4.7 started by root, uid 0
May 25 21:25:05 dbsvr pppd[5414]: PPP session is 1
May 25 21:25:05 dbsvr pppd[5414]: Connected to 10:e8:78:4c:88:4b via interface mv2
May 25 21:25:05 dbsvr pppd[5414]: Using interface ppp1
May 25 21:25:05 dbsvr pppd[5414]: Connect: ppp1 <--> mv2
May 25 21:25:08 dbsvr pppd[5414]: Remote message: Login ok
May 25 21:25:08 dbsvr pppd[5414]: PAP authentication succeeded
May 25 21:25:08 dbsvr pppd[5414]: peer from calling number 10:E8:78:4C:88:4B authorized
May 25 21:25:08 dbsvr pppd[5414]: local  IP address 184.97.41.242
May 25 21:25:08 dbsvr pppd[5414]: remote IP address 71.34.230.254
May 25 21:25:08 dbsvr pppd[5414]: LCP terminated by peer
May 25 21:25:08 dbsvr pppd[5414]: Connect time 0.0 minutes.
May 25 21:25:08 dbsvr pppd[5414]: Sent 0 bytes, received 0 bytes.
May 25 21:25:08 dbsvr pppd[5414]: Modem hangup
May 25 21:25:08 dbsvr pppd[5414]: Connection terminated.
May 25 21:25:08 dbsvr pppd[5414]: Failed to disconnect PPPoE socket: 114 Operation already in progress

Also not sure which device/interface the gateway address is to be assigned to.

?-)

markfoodyburton commented 5 years ago

I'm not using PPP modems - not sure macvlan helps for PPP (maybe, I've never tried). I suspect you are falling fowl of the way PPPoE works - it sends discovery packets, which then cause the currently active session to drop (at least, I had much the same problem myself). In the end, I ended up not using PPP (which makes setting the 'consumer' modems up a little easier too :-) )

legolas108 commented 5 years ago

OK, eventually got PPP connections to work (with hardware network interfaces), and a very stable ethernet bonding device also based on OpenVPN and FireHOL. Couldn't succeed with MLVPN. Thanks for all your help!

freemangl commented 4 years ago

OK, eventually got PPP connections to work (with hardware network interfaces), and a very stable ethernet bonding device also based on OpenVPN and FireHOL. Couldn't succeed with MLVPN. Thanks for all your help!

Hi, thanks for the guidance. I can't reply on "serverfault" , so I have to ask you here. I've been working on it for days and it never works. First of all, My network topology: https://i.imgur.com/yEj6JGM.png

In my scenario, please have a look of my config:

Client side:

vim /etc/network/interface

auto ens160
iface ens160 inet static
  address 192.168.50.254
  netmask 255.255.255.0
  gateway 192.168.50.1
  post-up /usr/local/bin/vpn-start
  pre-down /usr/local/bin/vpn-stop

Is my routing set up right in vpn-start?

# vim /usr/local/bin/vpn-start
#!/bin/bash
openvpn --config /etc/openvpn/tap0.conf
openvpn --config /etc/openvpn/tap1.conf
ip link add bond0 type bond
ip addr add 10.80.0.2/30 dev bond0

ip link set tap0 master bond0
ip link set tap1 master bond0
ip link set bond0 up mtu 1440
ip route add 192.168.50.0/24 via 10.80.0.1  <--Is this right?

How about the firewall config?

version 6

lan="ens160"
vpn="bond+"

interface "${lan}" lan
  policy accept

interface "${vpn}" vpn
  policy accept

router vpn2lan inface "${vpn}" outface "${lan}"
  policy accept

Server side:

tap0.conf:

# disable encryption, traffic continues unencrypted anyways
auth none
cipher none
dev tap0
mode p2p
port 1194
local 192.168.0.250   <-- Local ip or public ip?
proto udp
log /var/log/tap0.log
verb 3
ping 2
ping-restart 10
persist-tun
compress lz4-v2
daemon

Firewall config:

version 6
server_vpn_ports="udp/1194-1195"
client_vpn_ports="default"
snat4 to 192.168.0.250 outface ens33 dst not 192.168.0.250  <-- What does this mean? local ip or public ip?
interface ens33 web
  protection strong
  server ssh accept
  server vpn accept
  # more servers here as per your needs
  client all accept
interface bond0 vpn
  policy accept
router4 web2vpn inface ens33 outface bond0 dst 192.168.50.0/24,10.80.0.2  <--Is this right?
  client all accept

After I did all the setup, the link appeared to be established:

Client side:

root@ubcloud:~# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: load balancing (round-robin)
MII Status: down
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Client tap1.log:

Sat Aug  1 06:44:39 2020 ******* WARNING *******: All encryption and authentication features disabled -- All data will be tunnelled as clear text and will not be protected against man-in-the-middle changes. PLEASE DO RECONSIDER THIS CONFIGURATION!
Sat Aug  1 06:44:39 2020 Preserving previous TUN/TAP instance: tap1
Sat Aug  1 06:44:39 2020 TCP/UDP: Preserving recently used remote address: [AF_INET]118.152.247.193:1195
Sat Aug  1 06:44:39 2020 Socket Buffers: R=[212992->212992] S=[212992->212992]
Sat Aug  1 06:44:39 2020 UDP link local (bound): [AF_INET]192.168.50.254:1195
Sat Aug  1 06:44:39 2020 UDP link remote: [AF_INET]118.152.247.193:1195
Sat Aug  1 06:44:39 2020 Peer Connection Initiated with [AF_INET]118.152.247.193:1195
Sat Aug  1 06:44:41 2020 Initialization Sequence Completed

ping tunnel ip from client:

root@ubcloud:~# ping 10.80.0.2
PING 10.80.0.2 (10.80.0.2) 56(84) bytes of data.
64 bytes from 10.80.0.2: icmp_seq=1 ttl=64 time=0.019 ms
64 bytes from 10.80.0.2: icmp_seq=2 ttl=64 time=0.025 ms
64 bytes from 10.80.0.2: icmp_seq=3 ttl=64 time=0.026 ms
^C
--- 10.80.0.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2050ms
rtt min/avg/max/mdev = 0.019/0.023/0.026/0.005 ms
root@ubcloud:~# ping 10.80.0.1
PING 10.80.0.1 (10.80.0.1) 56(84) bytes of data.
From 10.80.0.2 icmp_seq=1 Destination Host Unreachable
From 10.80.0.2 icmp_seq=2 Destination Host Unreachable
From 10.80.0.2 icmp_seq=3 Destination Host Unreachable

Server side:

root@ubcloud:~# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: tap0
MII Status: up
Speed: 10 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: a6:fa:e9:55:79:2c
Slave queue ID: 0

Slave Interface: tap1
MII Status: up
Speed: 10 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: ae:b9:4a:50:b0:dc
Slave queue ID: 0

Server tap0.log:

root@ubcloud:~# cat /var/log/tap0.log
Sat Aug  1 06:43:27 2020 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
Sat Aug  1 06:43:27 2020 OpenVPN 2.4.9 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Oct 30 2019
Sat Aug  1 06:43:27 2020 library versions: OpenSSL 1.1.1  11 Sep 2018, LZO 2.08
Sat Aug  1 06:43:27 2020 ******* WARNING *******: All encryption and authentication features disabled -- All data will be tunnelled as clear text and will not be protected against man-in-the-middle changes. PLEASE DO RECONSIDER THIS CONFIGURATION!
Sat Aug  1 06:43:27 2020 TUN/TAP device tap0 opened
Sat Aug  1 06:43:27 2020 TUN/TAP TX queue length set to 100
Sat Aug  1 06:43:27 2020 Could not determine IPv4/IPv6 protocol. Using AF_INET
Sat Aug  1 06:43:27 2020 Socket Buffers: R=[212992->212992] S=[212992->212992]
Sat Aug  1 06:43:27 2020 UDPv4 link local (bound): [AF_INET]192.168.0.250:1194
Sat Aug  1 06:43:27 2020 UDPv4 link remote: [AF_UNSPEC]
Sat Aug  1 06:43:30 2020 Peer Connection Initiated with [AF_INET]172.227.224.139:1194
Sat Aug  1 06:43:30 2020 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
Sat Aug  1 06:43:30 2020 Initialization Sequence Completed

ping tunnel ip from server:

root@ubcloud:~# ping 10.80.0.1
PING 10.80.0.1 (10.80.0.1) 56(84) bytes of data.
64 bytes from 10.80.0.1: icmp_seq=1 ttl=64 time=0.048 ms
64 bytes from 10.80.0.1: icmp_seq=2 ttl=64 time=0.059 ms
^C
--- 10.80.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1030ms
rtt min/avg/max/mdev = 0.048/0.053/0.059/0.009 ms
root@ubcloud:~# ping 10.80.0.2
PING 10.80.0.2 (10.80.0.2) 56(84) bytes of data.
From 10.80.0.1 icmp_seq=1 Destination Host Unreachable
From 10.80.0.1 icmp_seq=2 Destination Host Unreachable
From 10.80.0.1 icmp_seq=3 Destination Host Unreachable

Any ideas what did I do wrong? How do I set up the routing on the client so that all the traffic in the LAN Let the traffic in the LAN go out from the server through the tunnel?

Thank you again to bring us such a wonderful solution!