search

openwrt / packages

Community maintained packages for OpenWrt. Documentation for submitting pull requests is in CONTRIBUTING.md
GNU General Public License v2.0
4.01k stars 3.49k forks source link

pbr: No default routes for IPv6 #21218

Open yuxincs opened 1 year ago

yuxincs commented 1 year ago

Maintainer: @stangri (find it by checking history of the package Makefile) Environment: (put here arch, model, OpenWrt version) X86 in a proxmox vm, openwrt 22.03.5, pbr version: 1.1.1-7 using nft

Description:

I'm running a wireguard client and server in openwrt, and I'm using pbr to only route my trusted network (192.168.2.0/24, fd00::1ced:cafe::20/64) through the VPN client, everything else can go through WAN.

So I added a policy to route 192.168.2.0/24 fc00:1ced:cafe::20/64 through vpn interface in the prerouting chain, and the ipv6 support is enabled. After this, I was able to get VPN working for ipv4 (tested on my dev machine in trusted zone), however, I couldn't access any ipv6 sites, and ping6 google.com hangs forever:

$ ping6 google.com
PING6(56=40+8+8 bytes) fd00:1ced:cafe:20:<RANDOM_SUFFIX> --> 2607:f8b0:4006:80b::200e

Then I ran /etc/init.d/pbr status on the openwrt host and saw

IPv6 table 257 route: unreachable default dev lo metric 1024 pref medium

So it seems that an ipv6 default route to the VPN client is not properly setup. I must admit that I don't have too much knowledge of ipv6 routing in general, but I'm more than happy to hack the script for further debugging if given a direction!

More background information:

Here's the entire output from /etc/init.d/pbr status, hope that helps!

Redacted information:

: public ipv4 from ISP : public ipv6 (/64) from ISP : VPN exit node public ipv4 : VPN exit node public ipv6 : the ULA (/128) given by VPN provider : link-local address ``` root@OpenWrt:~# /etc/init.d/pbr status ============================================================ pbr - environment pbr 1.1.1-7 running on OpenWrt 22.03.5. WAN (IPv4): wan/eth0/. WAN (IPv6): wan6/eth0/. ============================================================ Dnsmasq version 2.86 Copyright (c) 2000-2021 Simon Kelley Compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile ============================================================ pbr chains - policies chain pbr_forward { # handle 57 } chain pbr_input { # handle 58 } chain pbr_output { # handle 59 } chain pbr_prerouting { # handle 60 ip saddr @pbr_vpn_4_src_ip_cfg036ff5 goto pbr_mark_0x020000 comment "all" # handle 10256 ip6 saddr @pbr_vpn_6_src_ip_cfg036ff5 goto pbr_mark_0x020000 comment "all" # handle 10257 } chain pbr_postrouting { # handle 61 } ============================================================ pbr chains - marking chain pbr_mark_0x010000 { # handle 10244 counter packets 0 bytes 0 meta mark set meta mark & 0xff01ffff | 0x00010000 # handle 10245 return # handle 10246 } chain pbr_mark_0x020000 { # handle 10247 counter packets 2526 bytes 316824 meta mark set meta mark & 0xff02ffff | 0x00020000 # handle 10248 return # handle 10249 } ============================================================ pbr nft sets set pbr_vpn_4_src_ip_cfg036ff5 { # handle 10254 type ipv4_addr flags interval auto-merge comment "all" elements = { 192.168.2.0/24 } } set pbr_vpn_6_src_ip_cfg036ff5 { # handle 10255 type ipv6_addr flags interval auto-merge comment "all" elements = { fd00:1ced:cafe:20::/64 } } ============================================================ IPv4 table 256 route: default via dev eth0 IPv4 table 256 rule(s): 30000: from all fwmark 0x10000/0xff0000 lookup pbr_wan IPv6 table 256 route: default via dev eth0 proto static metric 512 pref medium unreachable default dev lo metric 1024 pref medium IPv6 table 256 rule(s): dev eth0 proto static metric 1024 pref medium via dev eth0 proto static metric 512 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium default via dev eth0 proto static metric 512 pref medium unreachable default dev lo metric 1024 pref medium IPv4 table 257 route: default via dev mullvadnyc1 IPv4 table 257 rule(s): 30001: from all fwmark 0x20000/0xff0000 lookup pbr_mullvadnyc1 IPv6 table 257 route: unreachable default dev lo metric 1024 pref medium IPv6 table 257 rule(s): dev mullvadnyc1 proto kernel metric 256 pref medium unreachable default dev lo metric 1024 pref medium ```
schm0 commented 1 year ago

You can add an ipv6 route by yourself in /etc/config/network Something like:

config route6
    option interface 'wgc0'
    option target '::/0'
    option metric '256'
    option table 'pbr_wgc0'

Adapt interface and table to your config. Make sure to set option masq6 '1' in your wg/vpn zone config. Make sure forwarding is allowed from your lan to wg/vpn zone. Setup your pbr rules as usual and it should work for both ipv4 and ipv6. No need to disable prefix delegation. pbr setups the proper rules but no default routing table, so you have to setup it by yourself as described above.

yuxincs commented 1 year ago

@schm0 Thanks a lot! that indeed fixed the issue!

Keeping this issue open for pbr to add default routing table

stangri commented 1 year ago

You can add an ipv6 route by yourself in /etc/config/network Something like:

config route6
  option interface 'wgc0'
  option target '::/0'
  option metric '256'
  option table 'pbr_wgc0'

Adapt interface and table to your config. Make sure to set option masq6 '1' in your wg/vpn zone config. Make sure forwarding is allowed from your lan to wg/vpn zone. Setup your pbr rules as usual and it should work for both ipv4 and ipv6. No need to disable prefix delegation. pbr setups the proper rules but no default routing table, so you have to setup it by yourself as described above.

@schm0 is it something you believe pbr should be adding to the tables it creates?

schm0 commented 1 year ago

Without a proper route this obviously can't work. PBR copies the routes already in place. For WAN/WAN6 this is no problem because there is usually already a "catchall" default route in place (0.0.0.0, ::/0) But for tunnel connections in combination with policy based routing this is most likely not the case because it would route all traffic through the tunnel and that defeats the purpose of policy based routing.

I don't know... PBR creates it own tables anyway.... Why not use 0.0.0.0, ::/0 for those?

The question is... why does the ipv4 pbr_wgc0 table has a 0.0.0.0 target but the ipv6 one has no ::/0 target?

stangri commented 1 year ago

Without a proper route this obviously can't work. PBR copies the routes already in place. For WAN/WAN6 this is no problem because there is usually already a "catchall" default route in place (0.0.0.0, ::/0) But for tunnel connections in combination with policy based routing this is most likely not the case because it would route all traffic through the tunnel and that defeats the purpose of policy based routing.

I don't know... PBR creates it own tables anyway.... Why not use 0.0.0.0, ::/0 for those?

The question is... why does the ipv4 pbr_wgc0 table has a 0.0.0.0 target but the ipv6 one has no ::/0 target?

Would something like this work then? Nevermind the Makefile changes, just the extra line in the init script?

schm0 commented 1 year ago

Can you explain what your intention with this change is? Going through all that code is a bit much. || ipv4_error=1 this should be ipv6_error=1 ?

I'm trying to figure out why the pbr tables for IPv4 do have a 0.0.0.0 target but the IPv6 don't have a ::/0 target. Did pbr copy the IPv4 route from the wan interface? Or where does this come from?

Isn't it just easier to let pbr create tables with 0.0.0.0 and ::/0 ? What is the actual reason to copy the routes over to the pbr route tables?

luneth commented 1 month ago

Bumping this issue.

In the meantime, creating a static route sort of works. Except, when reloading pbr I have to disable then re-enable the rule to get ipv6 connectivity again.