fichtner
commented
1 year ago Have you tried the development version? Possibly similar to https://github.com/opnsense/core/issues/6637
Closed Neurocranium closed 1 year ago
fichtner
commented
1 year ago Have you tried the development version? Possibly similar to https://github.com/opnsense/core/issues/6637
Neurocranium
commented
1 year ago Haven't tried the development version yet but had a look at the radv stuff just now. The issue sounds similar but I actually don't think they are related.
I only do DHCPv6+RA for my LAN interface, which has static IPs for both versions with the IPv6 coming from one /64 subnet out of the /48 assigned by my ISP. The router advertisements are configured to be "Assisted". The generated /var/etc/radvd.conf doesn't change whatsoever before/during/after a reconnect causing the issue. Always looks like this:
root@opnsense:~ # cat /var/etc/radvd.conf
# Automatically generated, do not edit
# Generated for DHCPv6 server lan
interface vtnet1 {
AdvSendAdvert on;
MinRtrAdvInterval 20;
MaxRtrAdvInterval 60;
AdvLinkMTU 1500;
AdvDefaultPreference medium;
AdvManagedFlag on;
AdvOtherConfigFlag on;
prefix 2a01:xxxx:yyyy:zzzz::/64 {
DeprecatePrefix on;
AdvOnLink on;
AdvAutonomous on;
};
RDNSS 2a01:xxxx:yyyy:zzzz::ea01 {
};
DNSSL intern.foo.bar {
};
};Running through rc.newwanipv6, the $ip = get_interface_ipv6($interface); function returns an empty IP array and then runs into the "Failed to detect IP" but this is correct based on the current code. I traced get_interface_ipv6 through it's flow in interfaces.inc and it basically ends in a call to interfaces_primary_address6($interface, null).
function interfaces_primary_address6($interface, $ifconfig_details = null)
{
$ifcfgipv6 = $networkv6 = $subnetbitsv6 = null;
if (interfaces_has_prefix_only($interface)) {
/* extend the search scope for a non-NA mode to tracking interfaces */
$interface = array_merge([$interface], array_keys(link_interface_to_track6($interface)));
}
foreach (interfaces_addresses($interface, false, $ifconfig_details) as $addr) {
if ($addr['family'] != 'inet6' || $addr['deprecated'] || $addr['tentative'] || $addr['alias'] || $addr['scope']) {
continue;
}
$networkv6 = gen_subnetv6($addr['address'], $addr['bits']) . "/{$addr['bits']}";
$subnetbitsv6 = $addr['bits'];
$ifcfgipv6 = $addr['address'];
break; /* all done */
}
return [ $ifcfgipv6, $networkv6, $subnetbitsv6 ];
}
interfaces_addresses yields the below content for my case (bit of print_r debugging) and therefore all potential address candidates are discarded either by being IPv4 or by being a link local address with scope = 1.
Array (
[aaa.bbb.ccc.ddd] => Array (
[address] => aaa.bbb.ccc.ddd
[alias] =>
[bind] => 1
[bits] => 32
[deprecated] =>
[tentative] =>
[family] => inet
[interface] => wan
[key] => aaa.bbb.ccc.ddd
[name] => pppoe0
[scope] =>
)
[fe80::5a9c:fcff:fe10:ffa5%pppoe0] => Array (
[address] => fe80::5a9c:fcff:fe10:ffa5
[alias] =>
[bind] => 1
[bits] => 64
[deprecated] =>
[tentative] =>
[family] => inet6
[interface] => wan
[key] => fe80::5a9c:fcff:fe10:ffa5%pppoe0
[name] => pppoe0
[scope] => 1
)
[fe80::508b:7fff:fe1f:15ed%pppoe0] => Array (
[address] => fe80::508b:7fff:fe1f:15ed
[alias] =>
[bind] => 1
[bits] => 64
[deprecated] =>
[tentative] =>
[family] => inet6
[interface] => wan
[key] => fe80::508b:7fff:fe1f:15ed%pppoe0
[name] => pppoe0
[scope] => 1
)
)
As far as I can tell, the code simply doesn't anticipate my providers odd way of setting up IPv6 with only the LL connection for routing of the traffic.
fichtner
commented
1 year ago The exact code you reference has been changed on the development version so that’s why I asked.
Although the report on the other ticket is different in scope the “cannot detect IP” is a clear commonality. 😉
Neurocranium
commented
1 year ago Oh, thanks for pointing that part out. :sweat_smile: Well, not enough time over the break to switch to the dev version but certainly enough time to read a bit of code. :laughing:
Looks like the dev code should work for my situation as long as I find a way to ensure interfaces_has_prefix_only($interfaces) returns true. This would give me the fe80:: address and let me pass the is_ipaddr check for sure. 👍
Although I probably would have to set the advanced DHCPv6 options for prefix only, while no DHCPv6 going on in my ISPs setup. But if that's the only price to pay, I'm willing to do that. Maybe some kind of "Default routing via link-local" as additional option, could be nice.
Regardless, I'm going to figure out how to run the dev build (and switch back) this evening and give this a try. :-)
AdSchellevis
commented
1 year ago Regardless, I'm going to figure out how to run the dev build (and switch back) this evening and give this a try. :-)
See "Release Type" in https://docs.opnsense.org/manual/updates.html#update-settings
Neurocranium
commented
1 year ago Happy to report that the dev build is working. 👍 Well, almost at least.
Here is what I got on a reconnect:
2023-07-04T22:38:56+02:00 opnsense opnsense-devel 78347 - [meta sequenceId="1"] /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for wan(wan)
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 78347 - [meta sequenceId="2"] /usr/local/etc/rc.linkup: ROUTING: entering configure using 'wan'
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 78347 - [meta sequenceId="3"] /usr/local/etc/rc.linkup: plugins_configure monitor (,WAN_PPPOE)
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 78347 - [meta sequenceId="4"] /usr/local/etc/rc.linkup: plugins_configure monitor (execute task : dpinger_configure_do(,WAN_PPPOE))
2023-07-04T22:38:57+02:00 opnsense kernel - - [meta sequenceId="5"] <6>ng0: changing name to 'pppoe0'
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 78347 - [meta sequenceId="6"] /usr/local/etc/rc.linkup: plugins_configure ipsec (,wan)
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 78347 - [meta sequenceId="7"] /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,wan))
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 78347 - [meta sequenceId="8"] /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 78347 - [meta sequenceId="9"] /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 93455 - [meta sequenceId="10"] /usr/local/etc/rc.newwanipv6: IP renewal starting (new: fe80::508b:7fff:fe1f:15ed%pppoe0, old: fe80::508b:7fff:fe1f:15ed%pppoe0, interface: WAN[wan], device: pppoe0, force: yes)
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 93455 - [meta sequenceId="11"] /usr/local/etc/rc.newwanipv6: ROUTING: entering configure using 'wan'
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 93455 - [meta sequenceId="12"] /usr/local/etc/rc.newwanipv6: ROUTING: configuring inet6 default gateway on wan
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 93455 - [meta sequenceId="13"] /usr/local/etc/rc.newwanipv6: ROUTING: setting inet6 default route to fe80::c664:13ff:feda:cf00%pppoe0
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 93455 - [meta sequenceId="14"] /usr/local/etc/rc.newwanipv6: plugins_configure monitor (,WAN_PPPOEV6)
2023-07-04T22:38:57+02:00 opnsense opnsense-devel 93455 - [meta sequenceId="15"] /usr/local/etc/rc.newwanipv6: plugins_configure monitor (execute task : dpinger_configure_do(,WAN_PPPOEV6))
2023-07-04T22:38:58+02:00 opnsense opnsense-devel 98239 - [meta sequenceId="16"] /usr/local/etc/rc.newwanip: IP renewal starting (new: aaa.bbb.ccc.ddd, old: aaa.bbb.ccc.ddd, interface: WAN[wan], device: pppoe0, force: yes)Afterwards both routes are set again and my connection is fully working.
Almost working, because I slightly had to modify the interfaces_has_prefix_only($interface) function in interfaces.inc to make it work.
function interfaces_has_prefix_only($interface)
{
$interfaces_a = config_read_array('interfaces');
return empty($interfaces_a[$interface]['adv_dhcp6_config_file_override']) &&
((!empty($interfaces_a[$interface]['adv_dhcp6_config_advanced']) &&
empty($interfaces_a[$interface]['adv_dhcp6_id_assoc_statement_address_enable'])) ||
isset($interfaces_a[$interface]['dhcp6prefixonly']) ||
($interfaces_a[$interface]['ipaddrv6'] == 'pppoev6'));
}By default only the dhcp6 related options make the interface being flagged as prefix only. Since my uplink requires PPPoEv6 I simply was not able to set any of those options for my wan interface and simply added the || ($interfaces_a[$interface]['ipaddrv6'] == 'pppoev6') to make it also return true for pppoev6 interfaces.
So, as long as there is an UI option added to control this or the check for pppoev6 interface officially added, the current state of the dev build works to solve my problem.
fichtner
commented
1 year ago Thanks for taking a closer look. Hmm, I think it’s impossible for us to know if the PPPoE provider gives a GUA or not, but I previously believed it would as the link local address cannot be given through PPPoE because it’s already there. But apparently your ISP delegates a link-local? Never saw this before to be honest. We might have to remove a few restrictions from rc.newwanipv6 IP detection or remove it altogether as it doesn’t seem to add much value nowadays for multiple reasons.
Cheers, Franco
Neurocranium
commented
1 year ago If it is of any deeper interest, I just took a packet capture on the parent interface of the PPP connection and could try to investigate this some more.
At a first glance all what is happening for IPv6 is first a exchange of Interfaces Identifiers via PPP IPV6CP for both my local interface and the one at my ISP.
Briefly after that my ISPs router sends out a Neighbor Advertisement with the Router flag set. Guess that is what then brings me my default route via the link local address.
After that no further setup of a GUA or delegation of prefixes happens.
I assume that, since my ISP is giving me a static /48 range for picking my addresses, they expect me to simply be able to configure the rest statically and route everything from this /48 my way. This static config also works quite nicely for example on my LAN end. There I picked a /64 from my /48 range, assigned a static IPv6 to the opnsense LAN interface, then set up Router Advertisements and DHCPv6 for LAN and everything works...
fichtner
commented
1 year ago @Neurocranium sorry I was out of office for a while.
Yes, I've previously debugged this with another user... PPPoEv6 on WAN and static IPv6 setup on LAN if the ISP allows it. So we can close this as a support ticket?
Cheers, Franco
Important notices
Before you add a new report, we ask you kindly to acknowledge the following:
Describe the bug
For the WAN uplink with my ISP (tal.de) I'm using PPPoE/PPPoEv6 to establish a IPv4/IPv6 link. During the connection initialization I get an IPv4 IP and default route assigned and for IPv6 only a default route via a link-local connection on the pppoe0 interface is established.
After a (re)boot or with clicking on "Connect" in the interface overview, this setups works without issues.
The bug occurs after any form of connection loss (e.g. DSL sync issues), which stops the PPPoE connection and removes the default routes. While the IPv4 part recovers nicely once the connection is back, the IPv6 default gateway is not re-added, killing all IPv6 connectivity from/to my networks.
On reconnect both scripts
/usr/local/etc/rc.newwanipand/usr/local/etc/rc.newwanipv6get triggered. The IPv4 version correctly runs into an IP renewal, which also causes a reconfiguration of the default route.2023-07-04T00:13:45+02:00 opnsense opnsense 97866 - [meta sequenceId="17"] /usr/local/etc/rc.newwanip: IP renewal starting (new: aaa.bbb.ccc.ddd, old: aaa.bbb.ccc.ddd, interface: WAN[wan], device: pppoe0, force: yes)The IPv6 version though, is not able to detect an IP on the WAN interface and simply aborts processing before any routes can be re-added.
2023-07-04T00:13:45+02:00 opnsense opnsense 97866 - [meta sequenceId="16"] /usr/local/etc/rc.newwanipv6: Failed to detect IP for WAN[wan]This abort behavior is caused by the below check in
rc.newwanipv6due to no IP being returned for my WAN interface. The interface only has link-local IPv6 addresses but those are actively not picked up in the corresponding function frominterfaces.inc.To Reproduce
Expected behavior
After a re-connection the link-local based IPv6 default route from PPPoEv6 should be added to opnsense again. This should IMHO be picked up by the
rc.newwanipv6script during the re-connect.Describe alternatives you considered
I modded the
rc.newwanipv6script and replaced the is_ipaddr check from line 67 with the following code:While this works as a smooth workaround for the missing default gateway issue, it is causing some weird race condition between the
rc.newwanipandrc.newwanipv6during the reconnect:So, the snippet is certainly far from being a proper fix for this. ¯\_(ツ)_/¯
Relevant log files ppp log
system log at reconnect
Additional context
My ISP is doing a somewhat odd setup for IPv6. They don't use DHCPv6, SLAAC or any prefix delegation whatsoever. Just this default route via link-local plus a fixed /48 address block statically routed in direction of my opnsens. A (German) tutorial how this should be set up on a common router can be found here.
Environment
OPNsense 23.1.10_1-amd64 FreeBSD 13.1-RELEASE-p7 OpenSSL 1.1.1u 30 May 2023 KVM virtualized machine