search

daeuniverse / dae

eBPF-based Linux high-performance transparent proxy solution.
GNU Affero General Public License v3.0
3.03k stars 191 forks source link

[Bug Report] Not working for LAN with ufw enabled even though configured appropriately #364

Closed juzeon closed 9 months ago

juzeon commented 9 months ago

Checks

Current Behavior

Not working for LAN with ufw enabled even though IP Forward is allowed.

Expected Behavior

Works with ufw enabled and configured appropriately.

Steps to Reproduce

  1. Set up a clash client:

    root@phoenix:~# ps aux | grep clash
root        3785  0.0  1.0 1271456 30336 ?       Ssl  Dec08   1:07 /root/clash/clash.meta-linux-amd64-compatible-v1.16.0 -d config/
root       41727  0.0  0.0   6608  2228 pts/4    S+   09:15   0:00 grep --color=auto clash
root@phoenix:~# curl -x socks5h://127.0.0.1:7890 ifconfig.me
92.118.*.* (redacted)

  2. Configure dae:

    root@phoenix:~# cat /usr/local/etc/dae/config.dae 
global{
    log_level: info
    wan_interface: ens160
    lan_interface: docker0
    auto_config_kernel_parameter: true
}
group {
    my_group{
            policy: fixed(0)
    }
}
routing{
pname(clash.meta-linux-amd64-compatible-v1.16.0) -> must_direct
    fallback: my_group
}
node{
    local:'socks5://127.0.0.1:7890'
}

  3. Configure ufw and ip forward rules (with some redundant configuration in my case):

    
root@phoenix:~# ufw version
ufw 0.36.1
Copyright 2008-2021 Canonical Ltd.
root@phoenix:~# ufw status verbose
Status: active
Logging: on (low)
Default: allow (incoming), allow (outgoing), allow (routed)
New profiles: skip

To Action From

28573/tcp ALLOW IN Anywhere
22/tcp ALLOW IN Anywhere
80/tcp ALLOW IN Anywhere
443/tcp ALLOW IN Anywhere
3306/tcp ALLOW IN Anywhere
8089/tcp ALLOW IN Anywhere
9000/tcp ALLOW IN Anywhere
9853/tcp ALLOW IN Anywhere
36000/tcp ALLOW IN Anywhere
12345 ALLOW IN Anywhere
53 ALLOW IN Anywhere
28573/tcp (v6) ALLOW IN Anywhere (v6)
22/tcp (v6) ALLOW IN Anywhere (v6)
80/tcp (v6) ALLOW IN Anywhere (v6)
443/tcp (v6) ALLOW IN Anywhere (v6)
3306/tcp (v6) ALLOW IN Anywhere (v6)
8089/tcp (v6) ALLOW IN Anywhere (v6)
9000/tcp (v6) ALLOW IN Anywhere (v6)
9853/tcp (v6) ALLOW IN Anywhere (v6)
36000/tcp (v6) ALLOW IN Anywhere (v6)
12345 (v6) ALLOW IN Anywhere (v6)
53 (v6) ALLOW IN Anywhere (v6)

Anywhere on eth2 ALLOW FWD Anywhere on ens160
Anywhere on docker0 ALLOW FWD Anywhere on ens160
Anywhere on ens160 ALLOW FWD Anywhere on docker0
Anywhere (v6) on eth2 ALLOW FWD Anywhere (v6) on ens160
Anywhere (v6) on docker0 ALLOW FWD Anywhere (v6) on ens160
Anywhere (v6) on ens160 ALLOW FWD Anywhere (v6) on docker0

root@phoenix:~# sysctl -p net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1 net.ipv4.ip_forward = 1 net.ipv6.conf.default.forwarding = 1 net.ipv6.conf.all.forwarding = 1


5. Test on host machine, which works fine:

root@phoenix:~# dae reload OK root@phoenix:~# curl ifconfig.me 92.118..(redacted) root@phoenix:~# service dae status ● dae.service - dae Service Loaded: loaded (/etc/systemd/system/dae.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2023-12-09 09:06:47 UTC; 11min ago Docs: https://ghproxy.vespertilio.tk/github.com/daeuniverse/dae Process: 40949 ExecStartPre=/usr/local/bin/dae validate -c /usr/local/etc/dae/config.dae (code=exited, status=0/SUCCESS) Main PID: 40954 (dae) Tasks: 13 (limit: 3373) Memory: 66.7M CPU: 7.414s CGroup: /system.slice/dae.service └─40954 /usr/local/bin/dae run --disable-timestamp -c /usr/local/etc/dae/config.dae

Dec 09 09:18:13 phoenix dae[40954]: level=info msg="Group "my_group" node list:" Dec 09 09:18:13 phoenix dae[40954]: level=info msg=" local" Dec 09 09:18:13 phoenix dae[40954]: level=info msg="pname routing: trim "clash.meta-linux-amd64-compatible-v1.16.0" to "clash.meta-linux> Dec 09 09:18:13 phoenix dae[40954]: level=info msg="Routing match set len: 2/64" Dec 09 09:18:13 phoenix dae[40954]: level=warning msg="[Reload] Stopped old control plane" Dec 09 09:18:13 phoenix dae[40954]: level=warning msg="[Reload] Serve" Dec 09 09:18:13 phoenix dae[40954]: level=warning msg="[Reload] Finished" Dec 09 09:18:31 phoenix dae[40954]: level=info msg="localhost:60910 <-> 223.5.5.5:53" _qname=ifconfig.me. dialer=local dscp=0 mac="00:0c> Dec 09 09:18:31 phoenix dae[40954]: level=info msg="localhost:60910 <-> 223.5.5.5:53" _qname=ifconfig.me. dialer=local dscp=0 mac="00:0c> Dec 09 09:18:31 phoenix dae[40954]: level=info msg="localhost:41564 <-> ifconfig.me:80" dialer=local dscp=0 ip="34.117.118.44:80" mac="0> root@phoenix:~#


6. Test on docker with failure:

root@phoenix:~# docker run --rm curlimages/curl:8.5.0 -v 1.1.1.1 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 Trying 1.1.1.1:80... 0 0 0 0 0 0 0 0 --:--:-- 0:00:10 --:--:-- 0 root@phoenix:~# docker run --rm curlimages/curl:8.5.0 -v ifconfig.me % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- 0:00:04 --:--:-- 0 Could not resolve host: ifconfig.me

Note that I have confirmed that there's no process occupy the udp/53 as I have disabled the systemd-resolve and configured a nameserver in /etc/resolv.conf. (lsof -i:53 -n returns an empty result.)

  1. Disable ufw and it works fine: 
    root@phoenix:~# ufw disable
Firewall stopped and disabled on system startup
root@phoenix:~# docker run --rm curlimages/curl:8.5.0 -v ifconfig.me
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Host ifconfig.me:80 was resolved.
* IPv6: 2600:1901:0:bbc3::
* IPv4: 34.117.118.44
*   Trying 34.117.118.44:80...
* Connected to ifconfig.me (34.117.118.44) port 80
> GET / HTTP/1.1
> Host: ifconfig.me
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< server: fasthttp
< date: Sat, 09 Dec 2023 09:49:54 GMT
< content-type: text/plain
< Content-Length: 14
< access-control-allow-origin: *
< via: 1.1 google
< 
{ [14 bytes data]
100    14  100    14    0     0     18      0 --:--:-- --:--:-- --:--:--    18
* Connection #0 to host ifconfig.me left intact
92.118.*.* (redacted)

Environment

Anything else?

This has been confirmed reproducible on two on my machines.

dae-prow[bot] commented 9 months ago

Thanks for opening this issue!

st0nie commented 9 months ago

add as follows to /etc/ufw/before*.rules

-A ufw-before-input -m mark --mark 0x8000000 -j ACCEPT
st0nie commented 9 months ago

and make sure unblocking dae port like 12345 in ufw

juzeon commented 9 months ago

It works. Thank you!

And I have to remind that if we would like to allow this for ipv6 it should be a -A ufw6-before-input -m mark --mark 0x8000000 -j ACCEPT line added to /etc/ufw/before6.rules.

st0nie commented 9 months ago

It works. Thank you!

And I have to remind that if we would like to allow this for ipv6 it should be a -A ufw6-before-input -m mark --mark 0x8000000 -j ACCEPT line added to /etc/ufw/before6.rules.

Thanks and having modifed the PR

st0nie commented 9 months ago

BTW, for firewall related issue, it would be better to provide firewall's blocking log such as:

sudo dmesg  | grep "UFW BLOCK"