Support RFC 4191 for multiple ISP connections

[brianjmurrell]

If I have an OpenWRT router with multiple IPv6 consumer Internet connections, each has it's own prefix which provides the hosts behind my OpenWRT router with multiple (source) address (they can use to send packets).

If the ISP connections are not equal, where for example perhaps one is fast and unmetered while another is slower and metered, my router should be able to indicate to the hosts using it which source address(es) to prefer to use to influence the ISP connection the router will send the packets.

It seems that RFC 4191 is suitable for this.

It would be nice if odhcpd could be configured with the RFC 4191 high, medium, and low values that RFC 4191 specifies for the prefixes being advertised corresponding to each of the ISP interfaces available.

[dedeckeh]

This can be achieved by defining multiple logical OpenWrt interfaces on a physical interface; for each logical interface a DHCP section is created which allows to set the route preference via the uci option ra_preference (low/medium/high) while you can control via prefix_filter which prefixes (and routes) are included in the RA message. See also https://git.openwrt.org/?p=project/odhcpd.git;a=blob;f=README;h=0c562e6faed472dadf8170a9e4e83f214cb054a8;hb=HEAD#l116 and https://git.openwrt.org/?p=project/odhcpd.git;a=commit;h=750e457e3000187b85906814a2529ede24775325

[brianjmurrell]

This can be achieved by defining multiple logical OpenWrt interfaces on a physical interface;

So would the physical interface in this case be the LAN facing interface, i.e br-lan?

Any chance of providing an example of such logical interfaces on a physical interface? I don't think I have seen that sort of thing being done before, or really understand what kind of logical interface to which you are referring. Do you mean virtual interfaces such as br-lan:0, br-lan:1, etc.?

for each logical interface a DHCP section is created which allows to set the route preference via the uci option ra_preference (low/medium/high) while you can control via prefix_filter which prefixes (and routes) are included in the RA message.

How well does this work where the ISP route prefix is not static and can change? I guess I can use the provider's entire IPv6 space in the prefix_filter such that that would always encompass whatever subnet they give me, right? Am I understanding how this filter works correctly with that assumption?

[brianjmurrell]

Ahhh. Perhaps a logical interface is of the form:

config interface 'lan'
    option type 'bridge'
    option ifname 'eth0.1'

in /etc/config/network, yes?

So I can define multiple of those for the same option ifname ...? I've never run across that sort of thing before. Do each need to have IP addresses (i.e. some kind of option proto, etc.), etc. configured for the additional ones?

How will having multiple lan logical interfaces affect which interface my LAN traffic is received on, particularly for traffic sniffing (i.e. tcpdump) and firewalling purposes?

[brianjmurrell]

What version of odhcpd is this prefix_filter available in? As an experiment on OpenWRT 18.06.1 I tried limiting the prefixes announced using:

config dhcp 'lan'
        option interface 'lan'
        option dhcpv6 'disabled'
        option ra 'server'
        option ra_management '1'
        list dns 'fd31:aeb1:48df::2'
        option prefix_filter '2001:1234::/32'

but I'm still seeing the prefixes for the other WAN connections in the RAs on hosts connected to br-lan.

As far as I can tell, the odhcpd in OpenWrt 18.06.1 should be 57f639e3b0288c2ec12cdfb8598a182293746a0a which should include the commit that has the prefix_filter in it.

In any case, I opened #120 about this since AFAICT, I am using the prefix_filter correctly.

[brianjmurrell]

Having resolved the initial issue in #120 and with prefix_filter now working (for a single prefix at least), and setting ra_preference set to high, (with only a single logical interface configured thus far) this is what the RAs coming from odhcpd look like:

$ sudo rdisc6 pc_bridge
Soliciting ff02::2 (ff02::2) on pc_bridge...

Hop limit                 :           64 (      0x40)
Stateful address conf.    :           No
Stateful other conf.      :           No
Mobile home agent         :           No
Router preference         :         high
Neighbor discovery proxy  :           No
Router lifetime           :         1800 (0x00000708) seconds
Reachable time            :  unspecified (0x00000000)
Retransmit time           :  unspecified (0x00000000)
 Source link-layer address: 6C:B0:CE:AA:BB:CC
 MTU                      :         1500 bytes (valid)
 Prefix                   : 2001:1234:abcd:a700::/64
  On-link                 :          Yes
  Autonomous address conf.:          Yes
  Valid time              :         2424 (0x00000978) seconds
  Pref. time              :         2424 (0x00000978) seconds
 Route                    : 2001:567:1a:123::/64
  Route preference        :         high
  Route lifetime          :     infinite (0xffffffff)
 Route                    : 2607:1234:5678:abcd::/56
  Route preference        :         high
  Route lifetime          :        85293 (0x00014d2d) seconds
 Route                    : 2001:1234:abcd:a700::/56
  Route preference        :         high
  Route lifetime          :         2424 (0x00000978) seconds
 Route                    : fd31:aeb1:48df::/48
  Route preference        :         high
  Route lifetime          :     infinite (0xffffffff)
 Recursive DNS server     : fd31:aeb1:48df::2
  DNS server lifetime     :         6000 (0x00001770) seconds
 from fe80::6eb0:ceff:feaa:bbcc

While I am only getting the one prefix I have set in the filter, should I really be getting route information for all of the prefixes, including the ones filtered out given that all of them are having their preference set to the value that should only be getting set for the 2001:1234:abcd:a700::/64 prefix.

[dedeckeh]

Filtering of the routes was not taken into account by the author of the patch. I will look into this as routes should be filtered out as well imo

[brianjmurrell]

I think it makes sense to filter the routers also.

But frankly, I'm becoming pessimistic that this mechanism is going to achieve my originally stated goal of influencing the downstream nodes into which address they should use as their source address. I'd love to be wrong about that, but I'm losing hope that that will actually work/happen.

That said, I think the routes should be filtered also, so we should leave this open for that.

[dedeckeh]

I think it makes sense to filter the routers also.

Meanwhile I've pushed a patch which fixes this issue https://git.openwrt.org/?p=project/odhcpd.git;a=commit;h=0314d58abefd76b44d4f191e90df2e7a2b09dacb; only in master at the moment

But frankly, I'm becoming pessimistic that this mechanism is going to achieve my originally stated goal of influencing the downstream nodes into which address they should use as their source address. I'd love to be wrong about that, but I'm losing hope that that will actually work/happen.

Due to the downstream nodes not taking into account the Default Router Preference as described in RFC4191 ?

That said, I think the routes should be filtered also, so we should leave this open for that.

[brianjmurrell]

Meanwhile I've pushed a patch which fixes this issue https://git.openwrt.org/?p=project/odhcpd.git;a=commit;h=0314d58abefd76b44d4f191e90df2e7a2b09dacb; only in master at the moment

Nice!

Due to the downstream nodes not taking into account the Default Router Preference as described in RFC4191 ?

Yeah. That router preference ends up in the routing table, not assigned to the prefix/address on the interface.

An example routing table entry with the preference would look like:

2001:1234:5678:a700::/56 via fe80::6eb0:ceff:fef5:1e4a dev pc_bridge proto ra metric 425 pref medium

so the medium, AFAICT, is only telling the kernel that fe80::6eb0:ceff:fef5:1e4a is medium preference for packets destined to 2001:1234:5678:a700::/56 despite it being the only route for that prefix. And if I had a:

2607:1234:5678:cd00::/56 via fe80::6eb0:ceff:fef5:1e4a dev pc_bridge proto ra metric 425 pref high

in the same routing table, that is only telling the kernel (likewise) that fe80::6eb0:ceff:fef5:1e4a is high preference (but again being the only route for that prefix) for packets destined to 2607:1234:5678:cd00::/56.

I don't think the kernel is going to look at those two entries and use the medium and high assigned to them to choose to use it's address in the (high) prefix 2607:1234:5678:cd00::/56 as it's source address.

I would love to be wrong, but it seems a stretch.

[brianjmurrell]

That said, I find it hard to believe I am the only person in the world with this problem such that there seems to be no actual solution to this problem as far as I have been able to find.

Certainly this problem exists in the IPv4 world also, but since NAT is being used in a big part of that world, NAT is also being used to allow admins to set policies on which of several possible upstream connections to target traffic at simply by NATting to the source address that met their policy.

I'm (absolutely) loathed to NAT IPv6 though given the world of pain NAT causes in IPv4 and that IPv6 frees us of that pain.

[dedeckeh]

That said, I find it hard to believe I am the only person in the world with this problem such that there seems to be no actual solution to this problem as far as I have been able to find.

Certainly this problem exists in the IPv4 world also, but since NAT is being used in a big part of that world, NAT is also being used to allow admins to set policies on which of several possible upstream connections to target traffic at simply by NATting to the source address that met their policy.

Deployment of IPv6 grows and more and more people get an IPv6 prefix but you're one of the few lucky people having multiple routable IPv6 prefixes :) I'm aware of internet drafts and research projects (https://www.ipv6council.be/IMG/pdf/10_ulouvain_ipv6council.pdf) related to IPv6 PA multihoming but I haven't seen any ready out of the box implementation yet. Neither did I see any patches from the research project in OpenWrt although they claim to use OpenWrt for their project

I'm (absolutely) loathed to NAT IPv6 though given the world of pain NAT causes in IPv4 and that IPv6 frees us of that pain.

Indeed let's not return to the painful experiences of NAT; we should learn from history ... Meanwhile I've backported the filtering of routes based on prefix to 18.06 https://github.com/openwrt/openwrt/commit/62ddfaff32a3a819fac4d8cdfdad012a6707e537

[brianjmurrell]

I am fine to close this. I think the functionality described here works as it should, although it does not solve the problem I have, AFAICT.

I don't think the route preference is having any impact on source address selection at the client level. Sadly.

I have resorted to simply not even advertising the slow/metered connection (using the functionality describe here) and statically assigning addresses to the few nodes in the network that need to use that connection.

For "high availability" (i.e. use the backup/slow/metered connection when the primary/fast/unmetered one becomes unavailable) I guess I will have to use the same mechanism (foolsm) that I use for IPv4 detection of the same situation and have foolsm's action scripts change the odhcpd config (as ugly as that is -- although I guess I can use uci to do that?) to stop advertising the impaired connection and start advertising the slow/metered (i.e. backup) connection.

Although, I'm not sure that will even be entirely ineffective. I don't recall what clients do when a router stops advertising a prefix/route -- whether they stop trying to use it immediately or if it simply becomes deprecated. I guess some experimentation will be needed to figure that out.

[marc-h38]

While I am only getting the one prefix I have set in the filter, should I really be getting route information for all of the prefixes,...

There is still Route Information Pollution wrongly duplicating on-link Prefixes BTW, I just filed https://bugs.openwrt.org/index.php?do=details&task_id=3056 about that.

[brianjmurrell]

@marc-h38 Thanks for the pointer. Very interesting.

[marc-h38]

@dedeckeh fixed the ROI/POI collision I reported in 3056 (thx!)

[TCB13]

That said, I find it hard to believe I am the only person in the world with this problem such that there seems to be no actual solution to this problem as far as I have been able to find.

Certainly this problem exists in the IPv4 world also, but since NAT is being used in a big part of that world, NAT is also being used to allow admins to set policies on which of several possible upstream connections to target traffic at simply by NATting to the source address that met their policy.

I'm (absolutely) loathed to NAT IPv6 though given the world of pain NAT causes in IPv4 and that IPv6 frees us of that pain.

I was just trying to use this as a silly way of overriding RA's DNS server...

My use case is: ISP router as gateway and wireless AP. Added a cheap ARM SBC to run DNS and DHCP. With IPv4 I can simply disable the DHCP server of the ISP router and the ARM/OpenWrt will provide DHCP/DNS. Now with IPv6 things are harder because the ISP router does SLAAC and advertises its own IPv6 address as DNS server (no way to change this).

So I tried this:

root@openwrt:~# uci show dhcp.lan
(...)
dhcp.lan.ra='server'
dhcp.lan.dns='fe80::1:---:---:a2'
dhcp.lan.ra_default='2'
dhcp.lan.ra_maxinterval='20'
dhcp.lan.ra_mininterval='10'
dhcp.lan.domain='lan'
dhcp.lan.ra_preference='high'

With Wireshark I can see those RA's broadcasted:

// ip.version == 6 && icmpv6.type == 134
Frame 53090: 142 bytes on wire (1136 bits), 142 bytes captured (1136 bits) on interface en0, id 0
Ethernet II, Src: MS-NLB-PhysServer-xx_xx:xx:00:a2 (xx:xx:xx:xx:00:a2), Dst: IPv6mcast_01 (33:33:00:00:00:01)
Internet Protocol Version 6, Src: fe80::1:---:---:a2, Dst: ff02::1
Internet Control Message Protocol v6
    Type: Router Advertisement (134)
    Code: 0
    Checksum: 0x4911 [correct]
    [Checksum Status: Good]
    Cur hop limit: 64
    **Flags: 0x48, Other configuration, Prf (Default Router Preference): High**
    Router lifetime (s): 0
    Reachable time (ms): 0
    Retrans timer (ms): 0
    ICMPv6 Option (Source link-layer address : xx:xx:xx:xx:00:a2)
    ICMPv6 Option (MTU : 1500)
    **ICMPv6 Option (Recursive DNS Server fe80::1:---:---:a2)**
    ICMPv6 Option (DNS Search List Option lan)
    ICMPv6 Option (Advertisement Interval : 20000)

As you can see the OpenWrt is advertising itself fe80::1:---:---:a2 as a DNS server with high priority. Now the results:

• Microsoft Windows: actually adds my OpenWrt device to the list of DNS servers, but unfortunately to the bottom of the list. Is it ignoring the Router Preference): High option??
• macOS doesn't seem to do anything with those RAs.

I'm not sure if macOS isn't ignoring it all because of the Router lifetime (s): 0. That also seems broken because setting dhcp.lan.ra_lifetime='1800' doesn't seem to change the value.

Anyone with a similar use-case? Thank you.