search

opnsense / core

OPNsense GUI, API and systems backend
https://opnsense.org/
BSD 2-Clause "Simplified" License
3.36k stars 753 forks source link

21.7.8 sets invalid default route for IPv6 #5591

Closed coridonhenshaw closed 2 years ago

coridonhenshaw commented 2 years ago

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

Describe the bug

Since upgrading to 21.7.7, OPNsense consistently sets the LAN interface as the IPv6 default gateway, breaking IPv6 connectivity. Using the shell to set a default route to the ISP upstream router works around the problem.

I don't recall what version of OPNsense was in use before I upgraded, so I can't say which version is known-good. Upgrading to 21.7.8 did not correct the issue.

To Reproduce

Connect to an ISP which provides IPv6 with DHCP6-PD and configure OPNsense in the usual way (i.e. track6 on LAN interface, etc) All IPv6 addressing functionality will work as expected (prefix will be delegated, addresses will be assigned, and RAs will be broadcast to the LAN). The default route will, however, be wrong. Ping6 to external hosts will fail both from clients and the firewall itself.

Diagnosis via SSH (edited for brevity):

 LAN (vtnet0)    -> v4: 192.168.0.3/24
                    v6/t6: ****:****:****:****:5054:ff:fe14:e3a2/64
 WAN (vtnet1)    -> v4/DHCP4: ***.***.***.138/22
                    v6/DHCP6: fe80::5054:ff:fed8:92d2/64

root@OPNsense:~ # route -n -6 get default
   route to: ::
destination: ::
       mask: ::
    gateway: fe80::5054:ff:fe14:e3a2%vtnet1
        fib: 0
  interface: vtnet1
      flags: <UP,GATEWAY,DONE,STATIC>
 recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire
       0         0         0         0      1500         1         0

The default gateway address appears to be derived from the link local address for the LAN interface but uses the scope for the WAN interface.

Showing ISP RAs using tcpdump:

tcpdump -i vtnet1 -n -vv icmp6 and 'ip6[40] = 134'` 

tcpdump: listening on vtnet1, link-type EN10MB (Ethernet), capture size 262144 bytes
22:02:51.898024 IP6 (class 0xe0, hlim 255, next-header ICMPv6 (58) payload length: 24) fe80::a2f3:e4ff:fe7d:ca30 > fe80::5054:ff:fed8:92d2: [icmp6 sum ok] ICMP6, router advertisement, length 24
        hop limit 64, Flags [none], pref medium, router lifetime 4500s, reachable time 0s, retrans time 100s
          source link-address option (1), length 8 (1): a0:f3:e4:7d:cb:c5
            0x0000:  a0f3 e47d cbc5

fe80::a2f3:e4ff:fe7d:ca30 appears to be the ISP router.

Forcing a default gateway manually restores connectivity:

root@OPNsense:~ # route -6 del default
del net default
root@OPNsense:~ # route -6 add default fe80::a2f3:e4ff:fe7d:ca30%vtnet1
add net default: gateway fe80::a2f3:e4ff:fe7d:ca30%vtnet1
root@OPNsense:~ # ping6 google.ca
PING6(56=40+8+8 bytes) ****:****:****:****:5054:ff:fed8:92d2 --> 2607:f8b0:4004:83e::2003
16 bytes from 2607:f8b0:4004:83e::2003, icmp_seq=0 hlim=118 time=76.716 ms

Expected behavior

The default gateway should match that provided by ISP RA messages.

Describe alternatives you considered

Shell intervention is required to restore connectivity.

Screenshots

Not needed. The web UI provides the same gateway address as given by route -6

Relevant log files

From the general log, as shown by the web UI:

2022-02-21T22:43:54 opnsense[81798] /usr/local/etc/rc.routing_configure: ROUTING: keeping current default gateway 'fe80::5054:ff:fe14:e3a2%vtnet1'   
2022-02-21T22:43:54 opnsense[81798] /usr/local/etc/rc.routing_configure: ROUTING: setting IPv6 default route to fe80::5054:ff:fe14:e3a2  
2022-02-21T22:43:54 opnsense[81798] /usr/local/etc/rc.routing_configure: ROUTING: IPv6 default gateway set to wan    
2022-02-21T22:43:54 opnsense[81798] /usr/local/etc/rc.routing_configure: ROUTING: keeping current default gateway '142.179.120.1'    
2022-02-21T22:43:54 opnsense[81798] /usr/local/etc/rc.routing_configure: ROUTING: setting IPv4 default route to 142.179.120.1    
2022-02-21T22:43:54 opnsense[81798] /usr/local/etc/rc.routing_configure: ROUTING: IPv4 default gateway set to wan    
2022-02-21T22:43:54 opnsense[81798] /usr/local/etc/rc.routing_configure: ROUTING: entering configure using defaults

Additional context

Environment

OPNsense 21.7.8-amd64 FreeBSD 12.1-RELEASE-p22-HBSD OpenSSL 1.1.1m 14 Dec 2021

fichtner commented 2 years ago

@coridonhenshaw What's the contents of the following file?

# cat /tmp/vtnet1_routerv6

And what is your IPv6 WAN type set to?

Last but not least can you ping this?

# ping6 fe80::5054:ff:fe14:e3a2%vtnet1

Cheers, Franco

coridonhenshaw commented 2 years ago 
root@OPNsense:~ # cat /tmp/vtnet1_routerv6 
fe80::5054:ff:fe14:e3a2
root@OPNsense:~ # ping6 fe80::5054:ff:fe14:e3a2%vtnet1
PING6(56=40+8+8 bytes) fe80::5054:ff:fed8:92d2%vtnet1 --> fe80::5054:ff:fe14:e3a2%vtnet1
^C
--- fe80::5054:ff:fe14:e3a2%vtnet1 ping6 statistics ---
8 packets transmitted, 0 packets received, 100.0% packet loss
root@OPNsense:~ #
fichtner commented 2 years ago

The only thing that can give you this address is dhcp6c which may or may not talk to your LAN side DHCPv6.

Not sure what your WAN IPv6 type is set to still but I would suggest temporarily disabling the tracking on LAN IPv6 to test the theory that there is a loop in your network between WAN and LAN and they talk to each other.

Cheers, Franco

OPNsense-bot commented 2 years ago

This issue has been automatically timed-out (after 180 days of inactivity).

For more information about the policies for this repository, please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.

If someone wants to step up and work on this issue, just let us know, so we can reopen the issue and assign an owner to it.