Support ISPs that send IPv6 RA's without either M or O flags

[syserr0r]

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

• [X] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
• [X] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug

RA's from an ISP that do not specify M or O flags won't run the script passed to rtsold by the -M and -O flags.

This bug has been raised as a regression here: https://redmine.pfsense.org/issues/14072 and a related fix to rtsold was added here: https://reviews.freebsd.org/D39931

To Reproduce

Steps to reproduce the behavior:

1. Set-up a system running radvd in Router Only mode
2. Set-up OPNsense downstream of that system
3. Observe OPNsense fails to assign a gateway

To confirm further:

1. Run rtsold manually with the -D -f flags appended so sending router and interface can be seen from RAs
2. Observe OPNsense fails to assign a gateway even though RAs are seen by rtsold
3. Manually run /var/etc/rtsold_script.sh $interface_name "$upstream_gateway" with values populated from RA in rtsold output
4. Observe OPNsense now correctly has the gateway set-up

Expected behavior

OPNsense is able to assign set-up the gateway (for RA's that do not have any flags set)

Describe alternatives you considered

Convincing an ISP to alter their RA set-up is unlikely to work

Screenshots

RA from one of my upstream ISPs:

Relevant log files

rtsold: multiple interfaces found
rtsold: multiple interfaces found
rtsold: probing vmx1
rtsold: probing pppoe5
rtsold: probing pppoe0
rtsold: checking if vmx1 is ready...
rtsold: vmx1 is ready
rtsold: set timer for vmx1 to 1s
rtsold: link-layer address option has null length on pppoe5. Treat as not included.
rtsold: checking if pppoe5 is ready...
rtsold: pppoe5 is ready
rtsold: set timer for pppoe5 to 1s
rtsold: link-layer address option has null length on pppoe0. Treat as not included.
rtsold: checking if pppoe0 is ready...
rtsold: pppoe0 is ready
rtsold: set timer for pppoe0 to 1s
rtsold: New timer is 1s
rtsold: timer expiration on vmx1, state = 1
rtsold: set timer for vmx1 to 4s
rtsold: New timer is 0s
rtsold: received RA from fe80::6bc:9fff:fea8:7300 on vmx1, state is 2
rtsold: Processing RA
rtsold: ndo = 0x357eb83f1be0
rtsold: ndo->nd_opt_type = 1
rtsold: ndo->nd_opt_len = 1
rtsold: rsid = [vmx1:slaac:[fe80::6bc:9fff:fea8:7300]]
rtsold: stop timer for vmx1
rtsold: New timer is 0s
rtsold: received RA from fe80::6bc:9fff:fea8:7300 on vmx1, state is 0
rtsold: Processing RA
rtsold: ndo = 0x357eb83f1be0
rtsold: ndo->nd_opt_type = 1
rtsold: ndo->nd_opt_len = 1
rtsold: rsid = [vmx1:slaac:[fe80::6bc:9fff:fea8:7300]]
rtsold: New timer is 0s
rtsold: received RA from fe80::6bc:9fff:fea8:7300 on vmx1, state is 0
rtsold: Processing RA
rtsold: ndo = 0x357eb83f1be0
rtsold: ndo->nd_opt_type = 1
rtsold: ndo->nd_opt_len = 1
rtsold: rsid = [vmx1:slaac:[fe80::6bc:9fff:fea8:7300]]
rtsold: New timer is 0s
rtsold: timer expiration on pppoe0, state = 1
rtsold: set timer for pppoe0 to 4s
rtsold: New timer is 0s
rtsold: New timer is 0s
rtsold: received RA from fe80::2a3:8eff:feca:ae80 on pppoe0, state is 2
rtsold: Processing RA
rtsold: ndo = 0x357eb83f1be0
rtsold: ndo->nd_opt_type = 5
rtsold: ndo->nd_opt_len = 1
rtsold: ndo = 0x357eb83f1be8
rtsold: ndo->nd_opt_type = 3
rtsold: ndo->nd_opt_len = 4
rtsold: rsid = [pppoe0:slaac:[fe80::2a3:8eff:feca:ae80]]
rtsold: stop timer for pppoe0
rtsold: timer expiration on pppoe5, state = 1
rtsold: set timer for pppoe5 to 4s
rtsold: New timer is 4s
rtsold: received RA from fe80::72e4:22ff:fed3:8c00 on pppoe5, state is 2
rtsold: Processing RA
rtsold: ndo = 0x357eb83f1be0
rtsold: ndo->nd_opt_type = 5
rtsold: ndo->nd_opt_len = 1
rtsold: ndo = 0x357eb83f1be8
rtsold: ndo->nd_opt_type = 3
rtsold: ndo->nd_opt_len = 4
rtsold: rsid = [pppoe5:slaac:[fe80::72e4:22ff:fed3:8c00]]
rtsold: stop timer for pppoe5
rtsold: there is no timer

Additional context

Killing the existing rtsold and re-running with the -D -f flags to keep in foreground with extra output shows it receives RA. Note that both my PPPoE ISP's RA contain flags so have their gateways assigned correctly. My non-PPPoE ISP (on vmx1) sends an RA without any flags and this RA is shown in the output but not handled

Taking the interface (vmx1) and sending router address (fe80::6bc:9fff:fea8:7300) from the above logs allows us to manually run the command that should be run automatically: /var/etc/rtsold_script.sh vmx1 "fe80::6bc:9fff:fea8:7300"

Running this sets up the gateway as expected and WAN IPv6 connectivity is established

Environment

OPNsense 23.7.1_3 (amd64, OpenSSL).

[fichtner]

Ok so we have a bug that’s in FreeBSD that I can’t find and was not explicitly mentioned by pfSense so I’m unsure what the issue (code line in rtsold) is and when it started?

The fix adds a new feature only. And it breaks the configuration behaviour requiring both a base and core fix…

I’m all for fixing this but I don’t see where this broke allegedly.

Cheers, Franco

[fichtner]

What baffles me is the technical analysis:

On 22.05, the O script was executed even when the RA didn't have M or O set.

It’s not true from what I know, at least from the FreeBSD perspective. We could never get an RA scripted that neither had M or O flag. And before that we could never get it scripted when it had no O flag (regardless of M). And this was years ago so I’m entirely unsure what broke and when unless pfSense patches their version and then forgot their patch that never made it to FreeBSD?

Cheers, Franco

[fichtner]

Yeah, found it. So they did forget their own patch. 🤦

https://github.com/pfsense/FreeBSD-src/commit/a66035374857c07c3ac27f4d168302304ef08798

[syserr0r]

I'm not sure what pfSense are doing, I only included it in case it helped (which it looks like it may not have done, sorry).

I'm just stuck with a WAN connection that requires manually entering a command/specifying the upstream link-local gateway to be able to have IPv6 function and would like it fixed.

I have raised this with the ISP but I suspect it unlikely they will change their RA settings for just me..

It doesn't apply to me but hypthetically could it be an RA has no flag if a link-local address to the gateway is used and DHCP is used only for PD?

[fichtner]

Looks like we went with what FreeBSD supports since 2018, for reference this was last seen here: https://github.com/opnsense/src/commit/194eea2899f3872370a849eecb2a2c34d13ba751

[fichtner]

I know one user where this is relevant, but having understood I’ll mark this upstream, merge the relevant commit and release on a normal time frame as this is not strictly a regression.

Cheers, Franco

[fichtner]

PS: As far as I understand the problem arises when you launch dhcp6c before dhclient is done you don’t get any RA M/O flag and so the IPv6 gateway stays empty.

[fichtner]

@syserr0r you are going to need 32a6a11 as well as the new binary for /usr/sbin/rtsold which I'm going to upload in a few minutes. Note that without the binary rtsold stops working completely.

# opnsense-patch 32a6a11
# cd /usr/sbin
# mv rtsold rtsold.old
# fetch https://pkg.opnsense.org/FreeBSD:13:amd64/snapshots/misc/rtsold
# chmod 555 rtsold

(make sure the binary works, not sure about permissions after download)

Cheers, Franco

[fichtner]

CC @maurice-w fun read for you :)

[syserr0r]

@syserr0r you are going to need 32a6a11 as well as the new binary for /usr/sbin/rtsold which I'm going to upload in a few minutes. Note that without the binary rtsold stops working completely.

# opnsense-patch 32a6a11
# cd /usr/sbin
# mv rtsold rtsold.old
# fetch https://pkg.opnsense.org/FreeBSD:13:amd64/snapshots/misc/rtsold

(make sure the binary works, not sure about permissions after download)

Cheers, Franco

Thank you for the speedy resolution 🙏 @fichtner, it is very reassuring after spending far too long trying to figure out the issue.

Patch and updated rtsold appear to work correctly (IPv6 gateway came up as expected without any interaction after disabling and re-enabling the interface). The only step I would add to anyone trying this is after running the commands above also run chmod 555 /usr/sbin/rtsold to ensure the permissions match the original.

Will I need to re-apply the patch after any updates before this goes live (or un-apply before updating)?

[fichtner]

Thanks, I updated the instructions to include the chmod for others. Thanks for testing!

Goes live in 23.7.2 so no extra care to be taken next week.

Cheers, Franco

[maurice-w]

I don't understand what this ISP is doing. There is no M or O flag, indicating there is no DHCPv6 server. But there doesn't seem to be a prefix in the RA either, so SLAAC isn't possible. How does address assignment work here? Or do you configure addresses / prefixes statically and only the gateway is configured dynamically?

[fichtner]

Context from one other case I know of: https://forum.opnsense.org/index.php?topic=13629.msg62864#msg62864