Support "track interface" for IPv6 on LAN against an SLAAC configured WAN

[liorokman]

My ISP ( 018.net.il ) provides native IPv6 support. The only stable way I could configure this was to select "SLAAC" on the WAN interface in the UI. Unfortunately for me, this means that I can't use "Track Interface" in the LAN configuration, since the Track Interface UI section doesn't support tracking interfaces which were configured via SLAAC.

I messed around the OPNSense configuration directly in the filesystem, and figured out that it is possible to track the configuration of an SLAAC configured interface with the following RADVD configuration:

interface em0 {
    AdvSendAdvert on;
    MinRtrAdvInterval 200;
    MaxRtrAdvInterval 600;
    AdvLinkMTU 1500;
    AdvDefaultPreference medium;
    prefix 2000::/64 {
        Base6Interface pppoe0;
        DeprecatePrefix on;
        AdvOnLink on;
        AdvAutonomous on;
        AdvRouterAddr on;
    };
    route ::/0 {
        RemoveRoute on;
    };
    DNSSL  home.internal  { };
};

When I manually change the radvd.conf to appear as above, and manually start radvd then the LAN interface tracks the IPv6 configuration on pppoe0 and the rest of my network gets valid IPv6 addresses and hosts are correctly routed to the Internet.

Since I am manually changing the radvd.conf file, every time I reboot the OPNSense machine the radvd.conf file reverts to not using the Base6Interface directive correctly and my LAN network stops configuring IPv6 correctly. Would it be possible to add support for the UI to generate an radvd.conf file as shown above? Effectively this would add support for "Track Interface" for SLAAC configured interfaces.

[fichtner]

hi @liorokman,

Assuming em0 is LAN, is there any documentation on the prefix of 2000::/64 available? I'm not sure if this is ISP agnostic.

Also what does your WAN and LAN IPv6 look like (obscured, doesn't have to be correct but just to get the hint), also part of 200:: ?

Cheers, Franco

[liorokman]

Hi @fichtner

The 2000::/64 range is not used on my local network. Specifying the Base6Interface option causes radvd to merge the subnet configured on the specified interface with the 2000::/64 range. As best as I could understand, this is done by OR-ing the range configured on pppoe0 (in my case using SLAAC) and 2000::/64. Since the IPv6 specification defines the Internet as any address in the 2000::/3 range, the result is that the correct range is actually used. See http://code.tools/man/5/radvd.conf/ for the description of the Base6Interface option.

For example, right now my WAN IPv6 address is 2a02:ed0:XXXX:XXXX:XXXX:XXXX:XXXX:f82c/64, and my LAN IPv6 then becomes 2a02:ed0:XXXX:XXXX:XXXX:XXXX:XXXX:8bf9 . Since SLAAC always configures /64 networks, and 2000::/64 | 2a02:ed0:XXXX:XXXX::/64 results in 2a02:ed0:XXXX:XXXX::/64, the local network is correctly configured.

-- Lior

[fichtner]

Hi Lior,

Aha, pretty cool! That sounds like it can be integrated without large effort. :)

Does this also work for a separate attached LAN network? Like if you had a em2 added to that config as well? Im trying to understand if there is some dynamic to how Prefix IDs are used in DHCPv6 mode or if we can simply track from as many interfaces as we want without further configuration.

Cheers, Franco

[liorokman]

Hi Franco,

Sorry, I have no idea about what would happen if there were an additional separate LAN. I don't have that kind of setup in order to test it.

-- Lior

[fichtner]

Hi Lior,

You can try this patch on top of 18.1.7:

 # opnsense-patch 53994b09

It allows you to track WAN from LAN even for SLAAC, and writes 2000::/64 prefix with Base6Interface. The actual Base6Interface could be wrong, but step by step. :)

Cheers, Franco

[liorokman]

/var/etc/radvd.conf is generated with the correct 2000::/64 and a Base6Interface.

However the actual interface is wrong - the file references em1 instead of pppoe0

Here's the file that was generated (slightly redacted):

# Automatically Generated, do not edit
# Generated config for slaac delegation from wan on lan
interface em0 {
    AdvSendAdvert on;
    MinRtrAdvInterval 3;
    MaxRtrAdvInterval 10;
    AdvLinkMTU 1500;
    AdvOtherConfigFlag on;
    prefix 2000::/64 {
        Base6Interface em1;
        AdvOnLink on;
        AdvAutonomous on;
        AdvRouterAddr on;
    };
    RDNSS 2620:0:ccc::2 2620:0:ccd::2 { };
    DNSSL home.internal { };
};

[fichtner]

@liorokman okay, this is kind of what I expected. Two questions here:

1. When you change Base6Interface to pppoe0 and reload radvd, does it work?
2. When you use SLAAC on WAN, do you get an IPv6 address?

I'm hoping you will say "yes" and "no", respectively. If that's true I know what to do to fix both. :)

[liorokman]

1. No, it doesn't. The difference seems to be that before this patch the LAN interface (em0) had its own IPv6 address, but now it doesn't anymore.
2. Yes it does. I can ping -6 google.com just fine from the opnsense CLI.

[liorokman]

I can get a working ipv6 configuration if I do the following:

1. Change the LAN interface ipv6 configuration to "Track Interface" and track the WAN interface
2. Copy the /var/etc/radvd.conf aside and fix it so that the Base6Interface is pppoe0
3. Change the LAN interface ipv6 configuration back to SLAAC
4. Copy the fixed radvd.conf from (2) back to /var/etc
5. Restart radvd

The change was to have the em0 interface autoconfigure itself with an IP from the range advertised by the radvd daemon.

[fichtner]

Hmm, hold on...let me rephrase the question to avoid confusion. Do you have 2000:: address on your WAN using SLAAC under any of these circumstances?

The point is that PPPoE is special that's why this mismatch happens with em0 and pppoe0 and there is no particular mode for SLAAC in place to allow it on top of a PPPoE yet.

[liorokman]

With the Base6Interface set to em1 I have an invalid 2000::/64 address configured on my internal LAN. I think this happens because em1 doesn't have a valid ipv6 address, so the Base6Interface directive merging doesn't work correctly.

If I change the Base6Interface to pppoe0, which has the correct SLAAC configured external ipv6 address, the router advertisements on my LAN become correct. I don't have any invalid 2000::/64 addresses in this case.

[liorokman]

In all of the above, the WAN interface itself doesn't have a 2000:: address. pppoe0 is always configured correctly with the range that my ISP advertised, while em1 has no ipv6 configuration.

[fichtner]

Lior,

Can you try 41e30403, under WAN: SLAAC it now has a checkbox configuration for "IPv4 connectivity". Check this and there should be an IPv6 on WAN and pppoe0 should appear in the radvd config?

# opnsense-patch 41e30403

Cheers, Franco

[liorokman]

Nearly there.

1. The radvd.conf file is now generated correctly, however the LAN interface doesn't get an IPv6 address yet. I think that the reason is that the LAN interface must be set to autoconfigure itself via SLAAC when it is tracking a SLAAC interface. "Track Interface" should on the LAN interface should both generate the radvd.conf file, and configure the LAN interface to SLAAC.
2. The patch doesn't cleanly apply, at least on top of the previous patch. Here's the single rejected block:

   # cat /usr/local/www/interfaces.php.rej 
   @@ -2551,7 +2558,7 @@
                          <td>
                            <input name="dhcp6usev4iface" type="checkbox" id="dhcp6usev4iface" value="yes" <?=!empty($pconfig['dhcp6usev4iface']) ? "checked=\"checked\"" : ""; ?> />
                            <div class="hidden" data-for="help_for_dhcp6usev4iface">
   -                              <?= gettext('Request the IPv6 prefix/information through the IPv4 PPP connectivity link.') ?>
   +                              <?= gettext('Request the IPv6 information through the IPv4 PPP connectivity link.') ?>
                            </div>
                          </td>
                        </tr>

[fichtner]

Yay! Yes, tracking WAN SLAAC from LAN does not yet correctly set LAN SLAAC yet.

Does your WAN also receive an IPv6 now?

Sorry about the reject, there are a couple of differences between the development version and 18.1.7 now.

[liorokman]

Other than the LAN SLAAC configuration - everything seems to be configured correctly at this point.

IPv6 works from the OPNSense host itself.

root@gateway:/var/etc # ping6 www.google.com
PING6(56=40+8+8 bytes) 2a02:ed0:XXXX:XXXX:XXXX:XXXX:XXXX:f82c --> 2a00:1450:4009:805::2004
16 bytes from 2a00:1450:4009:805::2004, icmp_seq=0 hlim=54 time=69.046 ms
16 bytes from 2a00:1450:4009:805::2004, icmp_seq=1 hlim=54 time=68.767 ms
^C
--- www.google.com ping6 statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 68.767/68.906/69.046/0.140 ms

[fichtner]

@liorokman okay, last step, we need to find out how /var/etc/dhcp6c_*.conf needs to be set up in order to work for you. Can you dump the contents of those files in your working condition?

[liorokman]

Sure.

Here's dhcp6_lan.conf:

interface em0 {
  information-only;
  request domain-name-servers;
  request domain-name;
  script "/var/etc/dhcp6c_lan_script.sh"; # we'd like some nameservers please
};

Here's dhcp6_wan.conf:

interface pppoe0 {
  information-only;
  request domain-name-servers;
  request domain-name;
  script "/var/etc/dhcp6c_wan_script.sh"; # we'd like some nameservers please
};

[liorokman]

When I manually run the following command, everything starts working:

ifconfig em0 inet6 accept_rtadv

[fichtner]

@liorokman thanks, this next patch adds SLAAC tracking, basically just adding accept_rtadv as requested:

# opnsense-patch 148056cd

Not sure if this is all just yet. In any case thank you for your help! :)

Cheers, Franco

[liorokman]

Applied, and seems to work across interface restarts.

I'll reboot the opnsense host later this evening, to see if it also works after a full reboot.

Thanks very much for this feature, and especially for your rapid response to my feature request!!!

-- Lior

[fichtner]

Sounds good. Fingers crossed. :)

[liorokman]

After a reboot there were a few issues:

1. There's a race condition between the radvd daemon startup and the pppoe connection. Radvd is started up before the ppp connection is established, leading to the 2000::/64 subnet advertised for a few seconds. Since the default value for AdvPreferredLifetime is 14400 seconds (4 hours), it means that any host which autoconfigures with the initial 2000::/64 will remain misconfigured for at least 4 hours, or until its interface is stopped and started.

2. Old routes are not removed when they become irrelevant, leading to no connectivity when the pppoe connection is actually established and the correct route starting to be advertised.

3. The radvd.conf generated after reboot doesn't contain the OpenDNS ipv6 addresses. Instead, the local IPv6 for em0 is advertised. Before I rebooted, the radvd.conf had advertised the OpenDNS DNS servers

I think adding the DeprecatePrefix on; option to the radvd configuration file (see example in the very first comment on this thread) will help with resolving the first two. I'm not sure about the third.

[fichtner]

1. and 2. should be fixed via 25381c0

# opnsense-patch 25381c0

3. Assuming you mean RDNSS this can only be caused by having Unbound or Dnsmasq enabled in the system?

[liorokman]

(1) and (2): I've applied the patch and will test it later today (3) : Yes, I have both Dnsmasq and the OpenDNS service enabled in the system.

[fichtner]

Hmm, for all intents and purposes Dnsmasq should use the OpenDNS servers as forwarders if you have "Allow DNS server list to be overridden by DHCP/PPP on WAN" disabled under System: Settings: General. The system does that because if it doesn't one could not use the Dnsmasq overrides for clients.

[liorokman]

I see your point.

In this case, then the problem is that Dnsmasq isn't responding on the ipv6 address to queries that originate from outside of the opnsense host. I can see in the netstat output that it is listening on port 53, and it even responds if I run a host lookup from the opnsense host itself

I also verified that the request reaches the opnsense host via tcpdump. Maybe dnsmasq needs to be refreshed once the pppoe connection is up?

[fichtner]

Could be. Does it help when you manually restart it?

# pluginctl dns

[fichtner]

Make sure that "Interfaces" under Services: Dnsmasq: Settings is not set to anything other than "All".

[liorokman]

I've verified that the Interfaces option is set to "All", and that according to netstat the listening socket is on udp6 *.53.

Restarting Dnsmasq with pluginctl dns didn't help.

Maybe this problem is for a different issue though - for now it's not that critical, since Dnsmasq is responding on ipv4, and resolving IPv6 lookups via the IPv4 socket.

[fichtner]

Okay, happy to look at this after this ticket scope is resolved. :)

[liorokman]

I just rebooted the opnsense host. There was one unexpected thing, and that is that the radvd.conf file didn't have 2000::/64 as the defined route, but instead it had the SLAAC configured IP defined.

It looked like this:

# Automatically Generated, do not edit
# Generated config for slaac delegation from wan on lan
interface em0 {
    AdvSendAdvert on;
    MinRtrAdvInterval 3;
    MaxRtrAdvInterval 10;
    AdvLinkMTU 1500;
    AdvOtherConfigFlag on;
    prefix 2a02:ed0:XXXX:XXXX::/64 {
        Base6Interface pppoe0;
        DeprecatePrefix on;
        AdvOnLink on;
        AdvAutonomous on;
        AdvRouterAddr on;
    };
    RDNSS 2a02:ed0:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX { };
    DNSSL home.internal { };
};

While I was expecting this:

# Automatically Generated, do not edit
# Generated config for slaac delegation from wan on lan
interface em0 {
    AdvSendAdvert on;
    MinRtrAdvInterval 3;
    MaxRtrAdvInterval 10;
    AdvLinkMTU 1500;
    AdvOtherConfigFlag on;
    prefix 2000::/64 {
        Base6Interface pppoe0;
        DeprecatePrefix on;
        AdvOnLink on;
        AdvAutonomous on;
        AdvRouterAddr on;
    };
    RDNSS 2a02:ed0:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX { };
    DNSSL home.internal { };
};

The problem here is that if the pppoe connection drops for any reason, then when the connection restarts the SLAAC configured network on pppoe changes. When that happens, then radvd.conf for the LAN connection is not updated, and the ipv6 network stops working.

[liorokman]

Could you please merge https://github.com/opnsense/core/pull/2401 ? I think it would fix the radvd.conf generation issue I saw.

[fichtner]

Whoopsie, your are right. Thanks!

[fichtner]

slightly modified the fix to merge the two conditionals into one for clarity: https://github.com/opnsense/core/commit/a6db9d

[liorokman]

Three reboots out of four attempts ended up with a working setup. The fourth had the same race condition as mentioned in (1) above - (https://github.com/opnsense/core/issues/2394#issuecomment-387151058) .

Maybe it's possible to disable SLAAC on the LAN interface that is tracking the WAN interface until the WAN interface is itself configured? That would mean running

ifconfig em0 inet6 -accept_rtadv

before em0 is started, and running

ifconfig em0 inet6 accept_rtadv

after pppoe0 is up and the IPv6 address on it is configured.

Is this possible?

[fichtner]

I am back on Monday. I think we can fix the reverse case easily as you suggested earlier.

Cheers, Franco

On 12. May 2018, at 10:43, liorokman notifications@github.com wrote:

Three reboots out of four attempts ended up with a working setup. The fourth had the same race condition as mentioned in (1) above - (#2394 (comment)) .

Maybe it's possible to disable SLAAC on the LAN interface that is tracking the WAN interface until the WAN interface is itself configured? That would mean running

ifconfig em0 inet6 -accept_rtadv before em0 is started, and running

ifconfig em0 inet6 accept_rtadv after pppoe0 is up and the IPv6 address on it is configured.

Is this possible?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[fichtner]

@liorokman sorry for the delay! maybe 0c1c48b will help with the race condition?

[liorokman]

I've applied the patch, I'll test it later tonight.

[fichtner]

@liorokman do you have good news? :)

[liorokman]

Sorry, I haven't tested this yet.

[liorokman]

After rebooting the LAN interface is configured incorrectly with an address in the 2000::/3 range. The WAN interface has a correct address created via SLAAC from the router advertisement provided by the ISP on pppoe0. The radvd.conf file is correct, but it seems as if radvd is not picking up on the fact that the pppoe0 configuration changed after radvd was started, so the advertisement actually sent (as viewed with radvdump) is wrong.

After manually restarting radvd the em0 interface received the correct address, and ipv6 started being routed correctly from other machines as well.

[fichtner]

Hi @liorokman,

Sorry for the delay!

Can you try this?

# opnsense-patch baf292e

Thank you, Franco

[liorokman]

Hi @fichtner

I'm not sure what was supposed to happen. I applied the patch, and then restarted the WAN interface. The router advertisement didn't change.

After looking at the code in radvd, it seems that radvd will re-read its configuration file when a SIGHUP is sent to it. Does the above do this?

kill -HUP `cat /var/run/radvd.pid`

Thanks, Lior

[fichtner]

It doesn't use HUP, but it will reconfigure and restart the service in the (hopefully) correct spot according to what you've told me. You should see the PID change during WAN interface restart?

Cheers, Franco

[liorokman]

The pid does indeed change during a WAN interface restart.

The problem is that radvd is restarted too soon. Since the ipv6 IP is generated due to SLAAC, the pppoe0 interface can only receive an IP address after a router advertisement has been received. In the few tries I made, this happened after radvd had already been restarted. The result was that the router advertisement for the internal network was wrong.

Is there any hook that can be used which occurs after an IPv6 address is changed on an interface? This would happen later than when the pppoe0 receives its IPv4 address.

-- Regards, Lior

[fichtner]

Hi Lior,

Can you try 3b7c0bc ? Switched over reload from IPv4 to IPv6. I initially thought it needs to track the physical PPPoE link but you are right about SLAAC being required to kick in.

Cheers, Franco

[liorokman]

Thanks.

I've applied the patch, I'll test it later - after XBox time is over :-)

-- Lior

[fichtner]

There was a bug in the last commit, this will fix it: ac362698a2

[liorokman]

I tried to apply ac36269 and it didn't apply correctly. The patch seemed so different than the original file, that I thought it was relevant to the latest opnsense release, so I upgraded the host to version 18.1.10.

Once I did that, the entire set of patches was removed, and they don't apply any more.

Could you rebase this branch on the latest version (18.1.10)?

-- Thanks, Lior