Closed RyuunoAelia closed 3 years ago
I write "working 6rd setup" because it was working sometime before the 20.1 upgrade (I don't know exactly when it stopped working, might be when performing a minor update, but since by definition I am dual-stack I did not realise until now).
I'm having the same issue, here's a little more detail from my end:
Describe the bug
Tested in 20.1 and 20.1.1 - 6RD on WAN interface doesn't work when configured through web UI.
It appears that the interface wan_stf
is created by checking the shell, but the default IPv6 route is not installed by default and the WAN IPv6 address/interface is not visible in the WebUI.
To Reproduce Steps to reproduce the behavior:
ifconfig
to check interface presence for wan_stf
pppoe0: flags=89d1<UP,POINTOPOINT,RUNNING,NOARP,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1492
inet6 fe80::210:18ff:fe78:1534%pppoe0 prefixlen 64 scopeid 0xd
inet 97.118.72.211 --> 207.225.112.1 netmask 0xffffffff
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
wan_stf: flags=4041<UP,RUNNING,LINK2> metric 0 mtu 1280
inet6 2602:61:7648:d300:: prefixlen 56
nd6 options=1<PERFORMNUD>
v4net 97.118.72.211/0 -> tv4br 205.171.2.64
groups: stf
[david@router ~]$ ping6 google.com
ping6: UDP connect: No route to host
netstat -rn
and Internet6:
section lacks default
entry
Internet6:
Destination Gateway Flags Netif Expire
::1 link#5 UH lo0
2602:61:7648:d300:: link#14 UHS lo0
2602:61:7648:d300::/56 link#14 U wan_stf
fe80::%bce1/64 link#2 U bce1
fe80::210:18ff:fe78:1536%bce1 link#2 UHS lo0
...
root@router:/home/david # route add -inet6 default -interface wan_stf
add net default: gateway wan_stf
netstat -rn
to confim presence of default Internet6 route
Internet6:
Destination Gateway Flags Netif Expire
default wan_stf US wan_stf
::1 link#5 UH lo0
2602:61:7648:d300:: link#14 UHS lo0
2602:61:7648:d300::/56 link#14 U wan_stf
fe80::%bce1/64 link#2 U bce1
...
root@router:/home/david # ping6 google.com
PING6(56=40+8+8 bytes) 2602:61:7648:d300:: --> 2607:f8b0:400f:800::200e
16 bytes from 2607:f8b0:400f:800::200e, icmp_seq=0 hlim=57 time=2.208 ms
16 bytes from 2607:f8b0:400f:800::200e, icmp_seq=1 hlim=57 time=3.124 ms
^C
--- google.com ping6 statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 2.208/2.666/3.124/0.458 ms
wan_stf
is not an interface in UI
14.2 WAN 6RD address is not listed until WAN interface
14.3 6RD gateway does not accurately show up in dashboard widget
Expected behavior Configure 6RD via WAN interface, and have the interface created, linked, default route installed, and appears on web UI automatically without terminal intervention.
Environment
OPNsense 20.1.1-amd64FreeBSD 11.2-RELEASE-p16-HBSD OpenSSL 1.1.1d 10 Sep 2019 Dell R210 ii
Just wanted to say that I'm having the same issue. Running route add -inet6 default -interface wan_stf fixes it.
what does the gateway overview look like? (System -> Gateways -> Single)
You can see here that it does have a gateway listed but it doesn't actually do anything until that command is run. It's almost like the GUI has the settings but they are not actually implemented.
maybe there's more detail in the log (system log usually), it's not something we can reproduce easily on our end at the moment.
So it appears the ipv4 gateway address was previously registered and is the same. However the ipv6 address appears to not be registered previously and is not being set on reboot.
Another thing I noticed is that the gateways dashboard correctly lists the gateway ip for ipv4 but not for the ipv6-rd.
The troubleshooting section in https://docs.opnsense.org/manual/gateways.html#troubleshooting contains some tips on how to debug gateway choices a bit more, if WAN_6RD is the one it should use, it advertises its dismissal in the logs (lack of address), which would explain why its not being used here.
but shouldn't the firewall be able to use IPv4 and IPv6 gateways simultaneously? I one have one valid gateway in IPv4, and one valid gateway in IPv6 through wan_stf, so shouldn't both be automatically selected?
also, looking at those /tmp
files:
[david@router ~]$ cat /tmp/wan_stf_defaultgwv6
2602:46:3b1b:9a00::205.171.2.64
[david@router ~]$ cat /tmp/wan_stf_routerv6
2602:46:3b1b:9a00::205.171.2.64
But you are correct:
pppoe0 has 3 files:
pppoe0_ip pppoe0_router pppoe0up
wan_stf has 2:
wan_stf_defaultgwv6 wan_stf_routerv6
Just wanted to say that I'm having the same issue. Running route add -inet6 default -interface wan_stf fixes it.
Same here. Centurylink Fiber Seattle, Wa. Trying for hours and finally this one line fixed everything. 20.1.6 -- fresh install.
If anyone finds this and is lost, this tutorial solved most of the riddles: https://bidetly.io/2020/03/20/centurylink-fiber-on-pfsense/
After that and the router command in a shell, devices are getting an ipv6. System:Gateways:Single still doesn't show any address for ipv6.
Also experiencing this problem (version 20.7). CenturyLink Fiber Boise, ID.
Executed "route add -inet6 default -interface wan_stf" in shell, then pings and test-ipv6.com work.
Happy to debug more, just need to know where to look.
I can confirm this issue as well, both on 20.1 and 20.7, running the "route add" command allows IPv6 to work for routing. However, as others have reported, it doesn't actively work in the UI: no WAN IPv6 in dashboard, gateway monitoring doesn't work, I've yet to test beyond this since my boxes with 6RD are in another city, and doing this all remote.
Still happening on 20.7. I created a script in /usr/local/etc/rc.syshook.d/start/
to hopefully get it to persist on reboot, but there's no gateway monitoring.
Confirmed also, same as above. Ran route add -inet6 default -interface wan_stf
and it works now.
This issue has been automatically timed-out (after 180 days of inactivity).
For more information about the policies for this repository, please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.
If someone wants to step up and work on this issue, just let us know, so we can reopen the issue and assign an owner to it.
Can we please NOT auto-close issues like this? That just pushes it out-of-sight, out-of-mind. This is a serious issue effecting a vast number of IPv6 users on OPNsense. Also, there has been considerable activity on this particular topic, the comments are full of people diving into the issue, finding temporary workarounds, and providing data input.
I agree. Just because the devs haven't been involved doesn't mean the community isn't here documenting the issue. Some of us would even be happy to put in a PR, but haven't because the devs haven't recognized the issue and made the necessary direction clear.
@AdSchellevis - I'll play with this, sounds similar the default route issue experienced I played with earlier. Do you want me to re-open with a new issue? I'd rather keep this one as it has some debugging information in it which I've already used to narrow down the possible problem.
@haukened - Can you clear your logs, reboot, so it's clean. Then post the system logs - obfuscate info if you feel you need to.
Our policy is clear and simple (and won't change), without an owner, issues time-out. It is absolutely useless to keep issues open if nobody bothers to work on them, even with a PR involved it won't guarantee the (core) team will spend time on an item, the software is free to use for everyone, peoples (limited) time obviously isn't.
For long standing discussions, best use our forum, which is available on https://forum.opnsense.org/
I agree. Just because the devs haven't been involved doesn't mean the community isn't here documenting the issue. Some of us would even be happy to put in a PR, but haven't because the devs haven't recognized the issue and made the necessary direction clear.
Let's be real. The one with a 6RD setup who can maintain it by adding real fixes to the code base will likely be the designated maintainer. I don't know anyone who has or will step up to this in the years we had brought 6RD back instead of letting it die completely. Assuming anyone is blocking free will in voluntary contributors mischaracterises the open source situation we are in here rather bluntly.
If it's your setup you fix it, you don't play the blame game against others and in the same sweep demotivate someone else from stepping in. Complaining never improved the situation, but to some it seems so because all they do is complain and someone fixes it for other reasons. This is called confirmation bias and it is bad for community interactions.
This ticket remains open until the next timeout period comes along in a couple of months. I expect a maintainer and a patch or at least less attitude towards a potential maintainer for "having not fixed this when someone else clearly asked for it".
Cheers, Franco
"If it's your setup you fix it" <<< That's EXACTLY what we've been trying to do, with a growing number of eyes trying to help diagnose it. But by closing the ticket, you're preventing more eyes from joining, and disenfranchising the eyes already working on it.
"don't play the blame game against others and in the same sweep demotivate someone else from stepping in" <<< isn't this EXACTLY what you're doing by not only closing this ticket out, but after re-opening it, giving us, the community, a deadline to solve it ourselves without even any sort of basic guidance?
I'll put up my offer here as well that I put up on Twitter. I can stand up an instance of OPNsense that I'm willing to give access to the core OPNsense dev team. I have a block of IPv4 addresses, some currently unused, that all have 6RD access. I've confirmed above that adding the static route is a temporary work-around, but this leaves inconsistencies with the Web UI, and doesn't persist reboots without adding a script to handle it.
If you check the entire thread here, we've nailed down the approximate time in the codebase that the change was made. We have a fairly clear idea on what core FreeBSD networking calls are missing. For someone that has deeper knowledge into the OPNsense stack, this shouldn't be that complicated of an issue to solve, there just needs to be proper registration of the IPv6 default gateway, that was lost sometime between 19.1 and 20.1 (so prior to the move to HardenedBSD 12)
@darkain let's be realistic, nobody is obliged to deliver (free) support on someones issues, it's free to use the software, free to ask for help as well, but if it takes too long for an issue to mature into a solution, we use our timeouts. As mentioned before, you can always use our forum for long running discussions.
If the rules of the game aren't clear, please read our contribution guidelines https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
Wow this got out of hand really quickly. @fichtner there was no blame game here, only a request for direction from the core dev's in guiding whether this was something that was wanted/needed. No is an acceptable answer. What I was looking for, I've gleaned from your response - you think that 6RD is old and should die, and it seems like you guys might have made that decision intentionally; as is your discretion. If that's the case, and you'd rather let old (and inferior) methods die and become unsupported - that is a valid direction. I'm uninterested in discussing people's views on confirmation bias, or assigning blame to anyone in the course of GitHub issues; its non-productive.
As @AdSchellevis mentions, nobody is obliged to provide anything, and I don't expect anyone to do any work for me unless they choose to. I simply wanted a direction from the core team on whether or not this was something they felt was worth fixing; as many of us are stuck on CenturyLink connections without another option (except maybe switching ISPs, but that's another argument altogether). Before a PR gets created and submitted, i think its prudent to solicit the feedback regarding whether a fix for this would be desired, or if it doesn't lie in the vision of the platform and should be left to die.
If someone analyses the issue and comes up with a PR I'm certainly willing to take a look and see if/how we can progress this into a workable solution for everyone. We don't have active plans removing 6rd (at the short term), but we don't have any plans debugging this either.
@RyuunoAelia you said it worked some time before 20.1. Any chance to test with 19.1.10 (or anyone else)?
I asked for the logs so I can see if this is appearing /usr/local/etc/rc.newwanipv6: ROUTING: setting IPv6 default route
I hope, this helps, which is an output from my log (OPNsense 20.7.3-amd64), when I started and got the Gateways assigned:
Oct 5 17:49:06 OPNsense opnsense[16662]: plugins_configure hosts ()
Oct 5 17:49:06 OPNsense opnsense[16662]: plugins_configure hosts (execute task : dnsmasq_hosts_generate())
Oct 5 17:49:06 OPNsense opnsense[16662]: plugins_configure hosts (execute task : unbound_hosts_generate())
Oct 5 17:49:06 OPNsense opnsense[24794]: /usr/local/etc/rc.routing_configure: ROUTING: entering configure using defaults
Oct 5 17:49:06 OPNsense opnsense[16662]: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'opt1'
Oct 5 17:49:07 OPNsense kernel: done.
Oct 5 17:49:06 OPNsense opnsense[24794]: /usr/local/etc/rc.routing_configure: ROUTING: IPv4 default gateway set to wan
Oct 5 17:49:06 OPNsense opnsense[24794]: /usr/local/etc/rc.routing_configure: ROUTING: setting IPv4 default route to 192.168.0.254
Oct 5 17:49:06 OPNsense opnsense[24794]: /usr/local/etc/rc.routing_configure: ROUTING: keeping current default gateway '192.168.0.254'
Oct 5 17:49:06 OPNsense opnsense[24794]: /usr/local/etc/rc.routing_configure: ROUTING: IPv6 default gateway set to wan
Oct 5 17:49:06 OPNsense opnsense[24794]: /usr/local/etc/rc.routing_configure: ROUTING: setting IPv6 default route to fe80::c225:6ff:feff:820d
Oct 5 17:49:07 OPNsense opnsense[16662]: /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to wan
Oct 5 17:49:07 OPNsense opnsense[16662]: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv4 default route
Oct 5 17:49:07 OPNsense opnsense[16662]: /usr/local/etc/rc.newwanip: ROUTING: IPv6 default gateway set to wan
Oct 5 17:49:07 OPNsense opnsense[16662]: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
Oct 5 17:49:07 OPNsense opnsense[16662]: plugins_configure monitor ()
Oct 5 17:49:07 OPNsense opnsense[16662]: plugins_configure monitor (execute task : dpinger_configure_do())
Oct 5 17:49:07 OPNsense opnsense[24794]: /usr/local/etc/rc.routing_configure: The WAN_DHCP6 monitor address is empty, skipping.
Oct 5 17:49:07 OPNsense kernel: done.
Oct 5 17:49:07 OPNsense opnsense[24794]: /usr/local/etc/rc.routing_configure: The WAN_DHCP monitor address is empty, skipping.
Oct 5 17:49:07 OPNsense opnsense[16662]: /usr/local/etc/rc.newwanip: The WAN_DHCP6 monitor address is empty, skipping.
Oct 5 17:49:07 OPNsense opnsense[16662]: /usr/local/etc/rc.newwanip: The WAN_DHCP monitor address is empty, skipping.
Oct 5 17:49:07 OPNsense kernel: .
Oct 5 17:49:07 OPNsense kernel: ..
Oct 5 17:49:07 OPNsense kernel: .
Oct 5 17:49:08 OPNsense kernel: ...
Oct 5 17:49:08 OPNsense kernel: done.
With this ongoing, I cannot ping my WAN gateway "fe80::c225:6ff:feff:820d" from OPNsense (FW rules are in place), I had described this more detailed here https://forum.opnsense.org/index.php?topic=19018 but did not come to a solution.
So the row you (@marjohn56 ) are requesting is nearly included, but without "v6" at the end, there are always only calls with "/usr/local/etc/rc.newwanip", is that a copy paste error from the past? Just guessing.
@andreaslink-de - I assume you are trying a link-local ping on the WAN interface diagnostics, so your testing like this?
It does not really matter, where I do it. I can do the same way as you did (but I'm not using PPPoE).
Or via console (bce0 is my correct WAN interface):
root@OPNsense:~ # ping6 -c 1 'fe80::c225:6ff:feff:820d%bce0'
PING6(56=40+8+8 bytes) fe80::221:5eff:fec8:be88%bce0 --> fe80::c225:6ff:feff:820d%bce0
--- fe80::c225:6ff:feff:820d%bce0 ping6 statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss
root@OPNsense:~ # netstat -nr6
Routing tables
Internet6:
Destination Gateway Flags Netif Expire
default fe80::c225:6ff:feff:820d%bce0 UG bce0
::1 link#8 UH lo0
2a02:xxx:yyyy:fc00::/64 link#1 U bce0
2a02:xxx:yyyy:fc00:221:5eff:fec8:be88 link#1 UHS lo0
[...]
It always does not work, even though the default gateway seems to be set correctly with the link local.
But something is strange there, when I do e.g. a traceroute, then it suddenly seems to use the correct public IP as source and therefore also the public IP of the gateway and not referring back to the link local, which is announced as the default GW as it can been seen above. So something seems to be hidden there, it's again WAN and same interface.
See here doing the (abbreviated) traceroute to e.g. google:
root@OPNsense:~ # traceroute6 ipv6.google.com
traceroute6: Warning: ipv6.l.google.com has multiple addresses; using 2607:f8b0:4003:c16::66
traceroute6 to ipv6.l.google.com (2607:f8b0:4003:c16::66) from 2a02:xxx:yyyy:fc00:221:5eff:fec8:be88, 64 hops max, 20 byte packets
1 2a02:xxx:yyyy:fc00:c225:6ff:feff:820d 0.914 ms 0.465 ms 0.903 ms
2 2a02:2f0:0:72:: 8.955 ms 6.798 ms 17.695 ms
3 2a02:2f0:0:34:: 4.507 ms 6.284 ms 4.582 ms
4 2a02:2f0:4002::5d32:a0 5.140 ms 4.675 ms 7.530 ms
5 2001:4860:0:12e4::2 5.641 ms
2001:4860:0:12e5::4 29.627 ms
[...]
14 2001:4860::cc:4001:8240 118.343 ms *
2001:4860::22:0:b7ef 118.819 ms
15 2001:4860:0:1::75 118.621 ms *
2001:4860:0:1::37d 118.186 ms
16 * * *
[...]
22 * *
2607:f8b0:4003:c16::66 118.076 ms
I would also like to add that this recently effected me as well. I migrated from pfsense to opnsense a couple of weeks ago (fresh install of 20.7). And got effected by this when trying to setup 6rd for centurylink in CO. I'm still new opnsense and not sure how to get logs for this, but if someone is willing to provide some guidance I'm more than happy to help solve this.
This issue has been automatically timed-out (after 180 days of inactivity).
For more information about the policies for this repository, please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.
If someone wants to step up and work on this issue, just let us know, so we can reopen the issue and assign an owner to it.
Hi - I came across this issue in trying to figure this out myself. In order to hopefully continue conversation and figure out the root cause, I've started a forum post with all that I could find on the issue so far. I hope this is helpful! https://forum.opnsense.org/index.php?topic=20260.0
There is some progress here which might fix it for some: https://forum.opnsense.org/index.php?topic=20260.msg98380#msg98380
This has actually been solved in 21.1, I suggest everyone who still faces this to update, try again and report if it doesn't work.
On 21.1, the route is properly being added now. ICMP/ping/traceroute are all working, however TCP connections from the router itself are not. This breaks pkg, which breaks the entire OPNsense update process. I have to disable 6RD to get updates now, pkg (or even just fetch) hangs forever waiting for data. I've yet to enable Track Interface on LAN due to the router itself having all of these weird issues.
This has been confirmed on two different installs now, both using CenturyLink with static public IPv4 WAN addresses.
On 20.7 with manually adding the gateway, things worked just fine.
Hi @darkain - I don't have that issue, everything including TCP V6 connections work from my 21.1 system now (I confirmed this on the forum post I started above, along with help from @franco and others). That forum post may be a better place to continue conversation, too.
I don't think this issue had any direct implications above Layer 3 - if you can successfully route ICMP6 (ping6) / UDP6 (traceroute - assuming default usage/flags) I would maybe double check your firewall rules, or DNS?
I just double-checked on my 21.1 test instance, and I can successfully run curl -v6 https://ifconfig.co
which connects over IPv6, and reports the IP address configured on the WAN interface (in my case, wan_stf
). This behavior is the same if I have a inside (LAN) interface configured with IPv6 tracking, and if I only have the WAN interface configured for IPv6.
Somehow this issue is still there in latest 23.7.
Running route add -inet6 default -interface wan_stf
in shell fixed the issue for me.
root@OPNsense:~ # ping ipv6.google.com
ping: UDP connect: No route to host
root@OPNsense:~ # route add -inet6 default -interface wan_stf
add net default: gateway wan_stf
root@OPNsense:~ # ping -c 4 ipv6.google.com
PING6(56=40+8+8 bytes) 2400:d803:xxxx:xxxx:: --> 2404:6800:4003:c05::8a
16 bytes from 2404:6800:4003:c05::8a, icmp_seq=0 hlim=57 time=4.362 ms
16 bytes from 2404:6800:4003:c05::8a, icmp_seq=1 hlim=57 time=4.239 ms
16 bytes from 2404:6800:4003:c05::8a, icmp_seq=2 hlim=57 time=4.437 ms
16 bytes from 2404:6800:4003:c05::8a, icmp_seq=3 hlim=57 time=4.620 ms
--- ipv6.l.google.com ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 4.239/4.415/4.620/0.138 ms
Describe the bug
When setting up the 6rd config, the default inet6 route is not set. Setting it up manually works.
If I setup the correct route using:
c105:1d01 being the IP address of the border gateway 193.5.29.1 in hex (for those who have the same issue, and are wondering how I worked around the issue).
To Reproduce Steps to reproduce the behavior:
Note: I am using NPTv6 and a local ipv6 address as Virtual IP on the internal network side. Though if I setup the route manually I get full connectivity without issue.
Expected behavior 6rd setup should set the default route instead of having to set it up manually.
Environment OPNsense 20.1 (amd64, OpenSSL).