search

opnsense / core

OPNsense GUI, API and systems backend
https://opnsense.org/
BSD 2-Clause "Simplified" License
3.06k stars 699 forks source link

Wireguard interface not coming online after reboot - "required IPv4 interface address could not be found, skipping" #7468

Open lcasale opened 1 month ago

lcasale commented 1 month ago

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

Describe the bug

Some of my wireguard interfaces are not coming up on reboot with the below error message. Some time back, I was experiencing similar issues when using DNS endpoints but was able to fix it by using IP addresses instead. Now running into this problem. Appears as if some dependencies are not quite ready when Wireguard is starting.

2024-05-18T08:34:39-04:00 | Warning | wireguard | /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The required VPS1 IPv4 interface address could not be found, skipping.

I noticed this sometime during the 24.1.x versions but not sure exactly when it worked last.

To Reproduce

Steps to reproduce the behavior:

  1. Have a Wireguard interface setup similar to https://docs.opnsense.org/manual/how-tos/wireguard-client-mullvad.html#
  2. Restart Opnsense
  3. Interface will be down at time of boot, error message in Wireguard logs
  4. Manually disabling and enabling interface will bring the connection up right away

Expected behavior

Wireguard interfaces should start at boot

Screenshots instance peer interface

Environment

Software version used and hardware type if relevant, e.g.:

OPNsense 24.1.7-amd64 FreeBSD 13.2-RELEASE-p11 OpenSSL 3.0.13

AdSchellevis commented 1 month ago

The message relates to your gateway monitor, can you try to enable "Disable Host Route" on the gateway and test again?

lcasale commented 1 month ago

The message relates to your gateway monitor, can you try to enable "Disable Host Route" on the gateway and test again?

I gave it a try but I'm seeing the same result

Edit: Just to clarify you're referring to the gateway for the Wireguard connection, right?

lcasale commented 4 weeks ago

Maybe I was focused on the wrong line in the logs. Here are all the Wireguard logs from boot. I disabled "Disable Host Route" for the vps2/wg2 gateway but not the vps1/wg3 gateway. Both stay offline until I disable/re-enable each Wireguard interface.

2024-05-19T08:19:15-04:00   Notice  wireguard   wireguard instance vps1 (wg3) started   
2024-05-19T08:19:15-04:00   Warning wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The required VPS1 IPv4 interface address could not be found, skipping.    
2024-05-19T08:19:15-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: plugins_configure monitor (execute task : dpinger_configure_do(,VPS1))    
2024-05-19T08:19:15-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: plugins_configure monitor (,VPS1) 
2024-05-19T08:19:15-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: ROUTING: entering configure using 'opt13' 
2024-05-19T08:19:15-04:00   Error   wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The command '/sbin/route -q -n add '-4' '10.10.10.1' -iface 'wg3'' returned exit code '1', the output was ''  
2024-05-19T08:19:15-04:00   Error   wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The command '/sbin/ifconfig 'wg3' 'inet' '10.10.10.2/24' alias' returned exit code '1', the output was 'ifconfig: ioctl (SIOCAIFADDR): File exists'   
2024-05-19T08:19:15-04:00   Notice  wireguard   wireguard instance vps1 (wg3) can not reconfigure without stopping it first.    
2024-05-19T08:19:15-04:00   Notice  wireguard   wireguard instance vps2 (wg2) started   
2024-05-19T08:19:15-04:00   Warning wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The required VPS2 IPv4 interface address could not be found, skipping.    
2024-05-19T08:19:15-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: plugins_configure monitor (execute task : dpinger_configure_do(,VPS2))    
2024-05-19T08:19:15-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: plugins_configure monitor (,VPS2) 
2024-05-19T08:19:15-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: ROUTING: entering configure using 'opt9'  
2024-05-19T08:19:15-04:00   Error   wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The command '/sbin/ifconfig 'wg2' 'inet' '10.0.11.2/32' alias' returned exit code '1', the output was 'ifconfig: ioctl (SIOCAIFADDR): File exists'    
2024-05-19T08:19:15-04:00   Notice  wireguard   wireguard instance vps2 (wg2) can not reconfigure without stopping it first.    
2024-05-19T08:19:15-04:00   Notice  wireguard   wireguard instance ADblockVPN (wg0) started 
2024-05-19T08:19:15-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: ROUTING: entering configure using 'opt7'  
2024-05-19T08:19:15-04:00   Notice  wireguard   wireguard instance ADblockVPN (wg0) can not reconfigure without stopping it first.
AdSchellevis commented 4 weeks ago

The log suggests an overlapping address in your configuration, tried to add the sames address via a virtual ip as well?

2024-05-19T08:19:15-04:00   Error   wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The command '/sbin/ifconfig 'wg3' 'inet' '10.10.10.2/24' alias' returned exit code '1', the output was 'ifconfig: ioctl (SIOCAIFADDR): File exists'
lcasale commented 4 weeks ago

The log suggests an overlapping address in your configuration, tried to add the sames address via a virtual ip as well?

Not that I can tell. Here's a screen shot of my only VIP (which I think is used to access my modem's GUI) and the gateway settings. virtualip gateway

I searched around all the settings and I could only find "10.10.10.2/24" in the WG interface settings. Is there somewhere else I should be look?

If there was an IP conflict somewhere, wouldn't it prevent the WG instance from ever coming online? Once I disable and re-enable the WG instance it comes online immediately without any errors.

AdSchellevis commented 4 weeks ago

what does netstat -nr4 on clean boot? (you may remove the non relevant networks)

lcasale commented 4 weeks ago

this is with vps1/wg3 gateway "Disable Host Route" unchecked but checked for vps2/wg2's gateway. both fail to come up on clean boot.

root@OPNsense:~ # netstat -nr4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
10.0.11.1          link#20            UHS         wg2
10.0.11.2          link#20            UHS         wg2
10.10.10.0/24      10.10.10.1         UGS         wg3
10.10.10.1         link#19            UHS         wg3
lcasale commented 4 weeks ago

Hmm, after the clean boot and then disabling/re-enabling the two WG instances, this is the output...

root@OPNsense:~ # netstat -nr4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
10.0.11.1          link#19            UHS         wg2
10.0.11.2          link#19            UH          lo0
10.10.10.0/24      10.10.10.1         UGS         wg3
10.10.10.1         link#20            UHS         wg3

the interface changes for 10.0.11.2 and the gateway links#'s swap. not sure if that's indicitive of something wrong with my config.

AdSchellevis commented 4 weeks ago

and a clean boot without doing anything afterwards?

lcasale commented 4 weeks ago

Sorry, my first post was the clean boot logs

Clean Boot

root@OPNsense:~ # netstat -nr4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
10.0.11.1          link#20            UHS         wg2
10.0.11.2          link#20            UHS         wg2
10.10.10.0/24      10.10.10.1         UGS         wg3
10.10.10.1         link#19            UHS         wg3

Clean boot + cycling WG instances

root@OPNsense:~ # netstat -nr4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
10.0.11.1          link#19            UHS         wg2
10.0.11.2          link#19            UH          lo0
10.10.10.0/24      10.10.10.1         UGS         wg3
10.10.10.1         link#20            UHS         wg3
AdSchellevis commented 4 weeks ago

ah, you seem to be misusing far gateway here, in which case it's not able to configure the interface anymore..... disable "far gateway" and reboot the box.

lcasale commented 4 weeks ago

ah, you seem to be misusing far gateway here, in which case it's not able to configure the interface anymore..... disable "far gateway" and reboot the box.

Ah I misunderstood the purpose of far gateway when I initially set everything up. I disabled it for both gateways and rebooted but still facing the issue.

2024-05-21T06:37:39-04:00   Notice  wireguard   wireguard instance vps1 (wg3) started   
2024-05-21T06:37:39-04:00   Warning wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The required VPS1 IPv4 interface address could not be found, skipping.    
2024-05-21T06:37:39-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: plugins_configure monitor (execute task : dpinger_configure_do(,VPS1))    
2024-05-21T06:37:39-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: plugins_configure monitor (,VPS1) 
2024-05-21T06:37:39-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: ROUTING: entering configure using 'opt13' 
2024-05-21T06:37:39-04:00   Error   wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The command '/sbin/route -q -n add '-4' '10.10.10.1' -iface 'wg3'' returned exit code '1', the output was ''  
2024-05-21T06:37:39-04:00   Error   wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The command '/sbin/ifconfig 'wg3' 'inet' '10.10.10.2/24' alias' returned exit code '1', the output was 'ifconfig: ioctl (SIOCAIFADDR): File exists'   
2024-05-21T06:37:39-04:00   Notice  wireguard   wireguard instance vps1 (wg3) can not reconfigure without stopping it first.    
2024-05-21T06:37:39-04:00   Notice  wireguard   wireguard instance vps2 (wg2) started   
2024-05-21T06:37:39-04:00   Warning wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The required VPS2 IPv4 interface address could not be found, skipping.    
2024-05-21T06:37:39-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: plugins_configure monitor (execute task : dpinger_configure_do(,VPS2))    
2024-05-21T06:37:39-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: plugins_configure monitor (,VPS2) 
2024-05-21T06:37:39-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: ROUTING: entering configure using 'opt9'  
2024-05-21T06:37:39-04:00   Error   wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The command '/sbin/ifconfig 'wg2' 'inet' '10.0.11.2/32' alias' returned exit code '1', the output was 'ifconfig: ioctl (SIOCAIFADDR): File exists'    
2024-05-21T06:37:39-04:00   Notice  wireguard   wireguard instance vps2 (wg2) can not reconfigure without stopping it first.    
2024-05-21T06:37:39-04:00   Notice  wireguard   wireguard instance ADblockVPN (wg0) started 
2024-05-21T06:37:39-04:00   Notice  wireguard   /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: ROUTING: entering configure using 'opt7'  
2024-05-21T06:37:39-04:00   Notice  wireguard   wireguard instance ADblockVPN (wg0) can not reconfigure without stopping it first.
DiSHTiX commented 3 weeks ago

Did you try disable the 'prevent interface removal'? I had a case were ovpn or wg couldnt reuse its interface when this option is enabled. To be honest, i see no difference with the option on or off.

Since then i removed all 'prevent IF removal' for all software interfaces like ovpn, wg, zerotier etc, i never had an issue again.

Disable prevent, and enable interface.. Confirm the interface is gone else reboot. Then enable it back only with enable interface leaving prevent off.

lcasale commented 3 weeks ago

Did you try disable the 'prevent interface removal'? I had a case were ovpn or wg couldnt reuse its interface when this option is enabled. To be honest, i see no difference with the option on or off.

Since then i removed all 'prevent IF removal' for all software interfaces like ovpn, wg, zerotier etc, i never had an issue again.

Disable prevent, and enable interface.. Confirm the interface is gone else reboot. Then enable it back only with enable interface leaving prevent off.

Interesting! I gave it a try but still running into the same issue.

lcasale commented 3 weeks ago

Ok so I tried enablling "Dynamic Gateway Policy" for the WG interfaces and deleted my own created gateways and it's working now. Is that expected behavior?

DiSHTiX commented 2 weeks ago

In your case, i believe the issue stems from having created a static gateway + defined a static gateway in wireguard. When the link comes up, Wireguard tries to add the gateway again but fails as it already exists, causing the error.

I assume you want to name your gateways appropriately. Well you can still rename dynamically created gateways by using the 'clone' function. After changing the name the old will disappear and your newly named version remains. If you wish to rename it later again you'll need to delete it, then use the clone again.