search

openwrt / packages

Community maintained packages for OpenWrt. Documentation for submitting pull requests is in CONTRIBUTING.md
GNU General Public License v2.0
3.98k stars 3.47k forks source link

mwan3: set option local_source 'lan' breaks DHCP #7397

Closed jsalatiel closed 5 years ago

jsalatiel commented 5 years ago

Maintainer: @feckert Environment: (openwrt 18.06.1 running on microtik )

Description: After enabling the "option local_source 'lan'" in the /etc/config/mwan3 my wired devices stop acknowledge the DHCP offer so they just do not get IP. ( Wireless devices somehow still get it )

See the log for two of my devices right after enabling option local_source lan:

DEVICE1:
Sun Nov 11 22:22:43 2018 daemon.info dnsmasq-dhcp[4062]: DHCPDISCOVER(br-lan) 44:37:19:14:a3:13
Sun Nov 11 22:22:43 2018 daemon.info dnsmasq-dhcp[4062]: DHCPOFFER(br-lan) 192.168.254.128 44:37:19:14:a3:13
Sun Nov 11 22:22:45 2018 daemon.info dnsmasq-dhcp[4062]: DHCPDISCOVER(br-lan) 44:37:19:14:a3:13
Sun Nov 11 22:22:45 2018 daemon.info dnsmasq-dhcp[4062]: DHCPOFFER(br-lan) 192.168.254.128 44:37:19:14:a1:13
.....

DEVICE2:

Sun Nov 11 22:24:33 2018 daemon.info dnsmasq-dhcp[4062]: DHCPDISCOVER(br-lan) ac:84:c6:5a:a5:38
Sun Nov 11 22:24:33 2018 daemon.info dnsmasq-dhcp[4062]: DHCPOFFER(br-lan) 192.168.254.167 ac:84:c6:5a:a5:38
Sun Nov 11 22:24:38 2018 daemon.info dnsmasq-dhcp[4062]: DHCPDISCOVER(br-lan) ac:84:c6:5a:a5:38
Sun Nov 11 22:24:38 2018 daemon.info dnsmasq-dhcp[4062]: DHCPOFFER(br-lan) 192.168.254.167 ac:84:c6:5a:a5:38
...

The moment i set the option local_source 'none'" , it works. See the log:

Sun Nov 11 22:29:15 2018 daemon.info dnsmasq-dhcp[4062]: DHCPDISCOVER(br-lan) 44:37:19:14:a3:13
Sun Nov 11 22:29:15 2018 daemon.info dnsmasq-dhcp[4062]: DHCPOFFER(br-lan) 192.168.254.128 44:37:19:14:a3:13
Sun Nov 11 22:29:15 2018 daemon.info dnsmasq-dhcp[4062]: DHCPREQUEST(br-lan) 192.168.254.128 44:37:19:14:a3:13
Sun Nov 11 22:29:15 2018 daemon.info dnsmasq-dhcp[4062]: DHCPACK(br-lan) 192.168.254.128 44:37:19:14:a3:13 owl

Sun Nov 11 22:30:54 2018 daemon.info dnsmasq-dhcp[4062]: DHCPDISCOVER(br-lan) ac:84:c6:5a:a5:38
Sun Nov 11 22:30:54 2018 daemon.info dnsmasq-dhcp[4062]: DHCPOFFER(br-lan) 192.168.254.167 ac:84:c6:5a:a5:38
Sun Nov 11 22:30:54 2018 daemon.info dnsmasq-dhcp[4062]: DHCPREQUEST(br-lan) 192.168.254.167 ac:84:c6:5a:a5:38
Sun Nov 11 22:30:54 2018 daemon.info dnsmasq-dhcp[4062]: DHCPACK(br-lan) 192.168.254.167 ac:84:c6:5a:a5:38 TL-SG105E

Is this the expected behaviour ?

feckert commented 5 years ago

This was fixed in the master with the following commit https://github.com/openwrt/packages/commit/46ed09408e554f182a688e0a7c47cdf85fc6c15d#diff-34590d11545c84b961d1992ce3aefdeb The option local_source should not be used anymore there are a lot of problems with them. On owrt-18.06 this should be set to none so you will not have fail over and load balancing on router initiate traffic. The fail over could be solved with an online metric see commit https://github.com/TDT-AG/packages/commit/be91e71805116ac1fd852a1ac0480737538d0b04 but this will get removed soon because on master "online metric" is not necessary anymore too. The scenario fail over and load balancing should work for router initiate traffic on the master and next stable out of the box.

jsalatiel commented 5 years ago

I have updated to mwan3 - 2.7.5-1. Now i have a problem that i still cant understand. If i try to bind to a specific interface in the router itself, the traffic is always going out on the default interface. Shouldn't it respect the interface where the app bount itself ?

Example:

Interface status:
 interface cabo is online and tracking is active   --> 20Mb link
 interface gvt is online and tracking is active      --> 10Mb link

# speedtest_cli.py   --source 192.168.1.111 (ip from the cabo interface - eth0.20)
Testing from Cabo Servicos De Telecomunicacoes Ltda (186.236.216.143)...
Testing download speed........................................
Download: 19.98 Mbit/s
Testing upload speed..................................................
Upload: 8.48 Mbit/s

#  speedtest_cli.py   --source 177.135.47.89 ( ip from the gvt interface)
Testing from Cabo Servicos De Telecomunicacoes Ltda (186.236.216.143)...
Testing download speed........................................
Download: 20.00 Mbit/s
Testing upload speed..................................................
Upload: 8.57 Mbit/s

Both connections are leaving through the cabo (eth0.20) interface ( external ip 186.236.216.143)

feckert commented 5 years ago

Shouldn't it respect the interface where the app bount itself ?

This has changed (@ptpt52). Router initiate traffic is now load_balanced as well. if you want this then you have to add a mwan3 rule and specify on which interface the traffic has to leave the router. What mwan3 config do you have?

jsalatiel commented 5 years ago

Thanks for the explanation! I have added the folowing rule:

config rule 'srcgvt'
    option sticky '0'
    option src_ip '177.135.47.89'
    option timeout '600'
    option use_policy 'gvt_cabo'

and it works now. It will prefer to use gvt(wan2), but fallback to cabo(wan1) if wan2 is unavailable.

Unfortunatelly the interface is pppoe with dynamic IP. is there a way to use something like:

    option src_ip_from 'gvt' ( interface name in /etc/config/network )

and the mwan3 script automatically detect the ip of the interface ?

Here it is my wan3 config file. I have replaced cabo with wan2 and gvt for wan1 to make easy to understand:

config globals 'globals'
    option mmx_mask '0x3F00'
    option rtmon_interval '5'
    option local_source 'none'

config interface 'wan2'
    list track_ip '8.8.8.8'
    list track_ip '208.67.222.222'
    option family 'ipv4'
    option reliability '2'
    option count '1'
    option timeout '2'
    option interval '5'
    option down '3'
    option up '8'
    option initial_state 'online'
    option track_method 'ping'
    option size '56'
    option max_ttl '60'
    option check_quality '0'
    option failure_interval '5'
    option recovery_interval '5'
    option flush_conntrack 'never'
    option enabled '1'

config interface 'wan1'
    list track_ip '8.8.8.8'
    list track_ip '208.67.220.220'
    option family 'ipv4'
    option reliability '1'
    option count '1'
    option timeout '2'
    option interval '5'
    option down '3'
    option up '8'
    option initial_state 'online'
    option track_method 'ping'
    option size '56'
    option max_ttl '60'
    option check_quality '0'
    option failure_interval '5'
    option recovery_interval '5'
    option flush_conntrack 'never'
    option enabled '1'

config member 'wan1_m1_w1'
    option interface 'wan1'
    option metric '1'
    option weight '1'

config member 'wan1_m2_w1'
    option interface 'wan1'
    option metric '2'
    option weight '1'

config member 'wan2_m1_w1'
    option interface 'wan2'
    option metric '1'
    option weight '1'

config member 'wan2_m2_w1'
    option interface 'wan2'
    option metric '2'
    option weight '1'

config policy 'wan1_only'
    list use_member 'wan1_m1_w1'

config policy 'wan2_only'
    list use_member 'wan2_m1_w1'

config policy 'balanced'
    list use_member 'wan1_m1_w1'
    list use_member 'wan2_m1_w1'

config policy 'wan1_wan2'
    list use_member 'wan1_m1_w1'
    list use_member 'wan2_m2_w1'

config policy 'wan2_wan1'
    list use_member 'wan2_m1_w1'
    list use_member 'wan1_m2_w1'

config rule 'srcwan1'
    option sticky '0'
    option src_ip '177.135.47.89'
    option timeout '600'
    option use_policy 'wan1_wan2'

config rule 'srcwan2'
    option sticky '0'
    option src_ip '192.168.111.1'
    option timeout '600'
    option use_policy 'wan2_wan1'

config rule 'common_ports'
    option sticky '1'
    option dest_port '22,80,443'
    option proto 'tcp'
    option use_policy 'wan2_wan1'

config rule 'default_rule'
    option dest_ip '0.0.0.0/0'
    option use_policy 'balanced'
jsalatiel commented 5 years ago

I did a little change in the mwan script so it accepts an interface as the source_ip. I am adding the diff here because it can help someone. There is probably a better way to do, but this code works for me.

diff -urN a/lib/mwan3/mwan3.sh b/lib/mwan3/mwan3.sh
--- a/lib/mwan3/mwan3.sh    2018-11-13 21:20:22.596184459 -0300
+++ b/lib/mwan3/mwan3.sh    2018-11-13 21:20:25.284196252 -0300
@@ -856,7 +856,7 @@

 mwan3_set_user_iptables_rule()
 {
-   local ipset family proto policy src_ip src_port sticky dest_ip dest_port use_policy timeout rule policy IPT
+   local ipset family proto policy src_ip src_port src_iface sticky dest_ip dest_port use_policy timeout rule policy IPT

    rule="$1"

@@ -875,6 +875,10 @@
        $LOG warn "Rule $1 exceeds max of 15 chars. Not setting rule" && return 0
    fi

+   . /lib/functions/network.sh
+   network_get_ipaddr src_iface $src_ip
+   [ -n "$src_iface" ] && src_ip="$src_iface" && $LOG info Using \'$src_ip\' as source ip for rule $rule.
+
    if [ -n "$ipset" ]; then
        ipset="-m set --match-set $ipset dst"
    fi

Now i can have failover and disable load balancing from traffic initiated from the router with the rules below:

config rule 'src_wan1'
    option src_ip 'wan1'
    option proto 'all'
    option sticky '0'
    option use_policy 'wan1_wan2'

config rule 'src_wan2'
    option src_ip 'wan2'
    option proto 'all'
    option sticky '0'
    option use_policy 'wan2_wan1'
ptpt52 commented 5 years ago

What do you want to achieve, mwan3 work perfect in master. local_source has no meaning can be dropped

jsalatiel commented 5 years ago

@ptpt52 i want to be able to force an application running inside the router to leave from an specific interface, but auto load balancing was not helping. I am closing this issue, the title is misleading. I was thinking that the local_source option was a way to disable load balancing from local traffic in the router itself.

jsalatiel commented 5 years ago

Thanks!

ptpt52 commented 5 years ago

if the application could bind interface, it should work. like ping -I eth0 but most application could not bind, it is better to change the application to do binding. https://stackoverflow.com/questions/14478167/bind-socket-to-network-interface

ptpt52 commented 5 years ago

You can also setup some mwan3 rule to archive the goal if your application access a specific target IP

config rule 'target'
    option proto 'all'
    option sticky '0'
    option use_policy 'wan_only'
    option dest_ip '1.1.1.1'

config member 'wan_only'
    option interface 'wan'
    option metric '1'
    option weight '1'
ptpt52 commented 5 years ago

There are many ways to adjust automatically according to the specific situation. ^_^

jsalatiel commented 5 years ago

There are many ways to adjust automatically according to the specific situation. ^_^

yeah =) mwan3 is amazing.

jsalatiel commented 5 years ago

if the application could bind interface, it should work. like ping -I eth0 but most application could not bind, it is better to change the application to do binding. https://stackoverflow.com/questions/14478167/bind-socket-to-network-interface

I know most applications cant bind, but you can always use the trick from https://www.x4b.net/kb/BindProcessToIPonLinux

ptpt52 commented 5 years ago

Here, the simple binding of the ip address is not effective, mwan3's balacned policy will change the outgoing interface. You need to bind the interface device like ping -I eth0 to get it work.

jsalatiel commented 5 years ago

Yeah, i noticed that! Unfortunately most applications wont allow bind to the interface, thats the reason i have patched the mwan3 script to allow use a rule to match the ip from the interface.

I did a little change in the mwan script so it accepts an interface as the source_ip. I am adding the diff here because it can help someone. There is probably a better way to do, but this code works for me.

diff -urN a/lib/mwan3/mwan3.sh b/lib/mwan3/mwan3.sh
--- a/lib/mwan3/mwan3.sh  2018-11-13 21:20:22.596184459 -0300
+++ b/lib/mwan3/mwan3.sh  2018-11-13 21:20:25.284196252 -0300
@@ -856,7 +856,7 @@

 mwan3_set_user_iptables_rule()
 {
- local ipset family proto policy src_ip src_port sticky dest_ip dest_port use_policy timeout rule policy IPT
+ local ipset family proto policy src_ip src_port src_iface sticky dest_ip dest_port use_policy timeout rule policy IPT

  rule="$1"

@@ -875,6 +875,10 @@
      $LOG warn "Rule $1 exceeds max of 15 chars. Not setting rule" && return 0
  fi

+ . /lib/functions/network.sh
+ network_get_ipaddr src_iface $src_ip
+ [ -n "$src_iface" ] && src_ip="$src_iface" && $LOG info Using \'$src_ip\' as source ip for rule $rule.
+
  if [ -n "$ipset" ]; then
      ipset="-m set --match-set $ipset dst"
  fi
Kurgan- commented 5 years ago

I have just run into the issue (dhcp stops working...) and I have worked around it by creating a new LAN2 interface with another (LAN) ip address. For example, my default LAN ip is 192.168.0.254, and I have set up a LAN2 interface (on the same phisical interface) with address 192.168.0.253. Then I tell mwan3 to use LAN2 as source for router generated traffic. This basically adds 192.168.0.253/32 to "lo" interface. This stops dhcp from breaking, and also makes mwan3 work properly.

this works on OpenWrt 18.06.1 r7258-5eb055306f / LuCI openwrt-18.06 branch (git-18.228.31946-f64b152)

feckert commented 5 years ago

This workaround should not be needed anymore for the next mwan3 release. This can already be tested in the master branch