search

opnsense / core

OPNsense GUI, API and systems backend
https://opnsense.org/
BSD 2-Clause "Simplified" License
3.07k stars 700 forks source link

rc.newwanipv6 does not exit #7534

Closed wevsty closed 1 week ago

wevsty commented 1 week ago

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

Describe the bug

After the system had been running for a while I noticed that the memory usage was abnormal, and after running ps -aux I could see a large number of rc.newwanipv6 related processes.

If processes that keep starting and don't quit eventually running out of memory.

I believe this does not have that issue on opnsense 24.1.7.

ps -aux output

USER      PID  %CPU %MEM    VSZ    RSS TT  STAT STARTED        TIME COMMAND
root       11 353.0  0.0      0     64  -  RNL  30May24 96574:38.30 [idle]
root    68934   1.3  0.0  23108   6396  -  Ss   18:58       0:00.00 /usr/local/sbin/ntpd -g -c /var/etc/ntpd.conf
root    21544   1.0  0.4 287184  71764  -  S    23:19       6:19.26 /usr/local/bin/python3 /usr/local/opnsense/scripts/unbound/logger.py (python3.11)
root    39620   0.9  1.5 662152 247920  -  S    Fri03     356:44.29 /usr/local/bin/python3 /usr/local/opnsense/service/configd.py console (python3.11)
root    71965   0.7  0.0  19500   7024  -  Ss   18:58       0:00.00 sshd: /usr/local/sbin/sshd [listener] 0 of 10-100 startups (sshd)
root    78632   0.7  0.0  13488   2436  -  S    18:58       0:00.00 /bin/sh /var/db/rrd/updaterrd.sh
root        0   0.6  0.0      0   1488  -  DLs  30May24   169:07.69 [kernel]
root    58486   0.6  0.0  13008   2072  -  Ss   18:58       0:00.00 /usr/local/sbin/filterlog -i pflog0 -p /var/run/filterlog.pid
root    77819   0.6  0.0  12724   1820  -  Ss   18:58       0:00.00 daemon: /var/db/rrd/updaterrd.sh[78632] (daemon)
root       12   0.2  0.0      0    336  -  WL   30May24   222:47.14 [intr]
dhcpd   36776   0.2  0.0  22444   7816  -  Ss   18:58       0:00.00 /usr/local/sbin/dhcpd -6 -user dhcpd -group dhcpd -chroot /var/dhcpd -cf /etc/dhcpdv6.conf -pf /var/run
root    44280   0.2  0.0  12620   1716  -  SC   18:58       0:00.00 sleep 20
root    37537   0.1  0.0  12724   1824  -  Ss   18:58       0:00.00 daemon: /usr/local/opnsense/scripts/dhcp/prefixes.sh[37657] (daemon)
root    37657   0.1  0.0  13488   2392  -  S    18:58       0:00.00 /bin/sh /usr/local/opnsense/scripts/dhcp/prefixes.sh
root        1   0.0  0.0  11376    376  -  SLs  30May24     2:14.04 /sbin/init
root        2   0.0  0.0      0     64  -  DL   30May24     0:00.00 [KTLS]
root        3   0.0  0.0      0     80  -  DL   30May24     0:00.00 [crypto]
root        4   0.0  0.0      0     32  -  DL   30May24     0:00.00 [cam]
root        5   0.0  0.0      0    928  -  DL   30May24     2:22.92 [zfskern]
root        6   0.0  0.0      0     16  -  DL   30May24     7:10.16 [pf purge]
root        7   0.0  0.0      0     16  -  DL   30May24     3:09.73 [rand_harvestq]
root        8   0.0  0.0      0     48  -  DL   30May24    14:01.79 [pagedaemon]
root        9   0.0  0.0      0     16  -  DL   30May24     8:47.89 [vmdaemon]
root       10   0.0  0.0      0     16  -  DL   30May24     0:00.00 [audit]
root       13   0.0  0.0      0     48  -  DL   30May24     0:00.02 [geom]
root       14   0.0  0.0      0     16  -  DL   30May24     0:00.00 [sequencer 00]
root       15   0.0  0.0      0     80  -  DL   30May24     0:26.27 [usb]
root       16   0.0  0.0      0     16  -  DL   30May24     0:17.73 [acpi_thermal]
root       17   0.0  0.0      0    128  -  DL   30May24     0:55.70 [bufdaemon]
root       18   0.0  0.0      0     16  -  DL   30May24     0:13.43 [syncer]
root       19   0.0  0.0      0     16  -  DL   30May24     0:09.11 [vnlru]
root       20   0.0  0.2  59820  26560  -  I    18:53       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root       21   0.0  0.1  59820  23708  -  I    17:42       0:00.20 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root       22   0.0  0.1  59820  23676  -  I    18:30       0:00.18 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root       28   0.0  0.1  59820  11944  -  I    09:55       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root       31   0.0  0.0      0     16  -  DL   30May24     0:00.24 [aiod1]
root       32   0.0  0.0      0     16  -  DL   30May24     0:00.25 [aiod2]
root       33   0.0  0.0      0     16  -  DL   30May24     0:00.24 [aiod3]
root       34   0.0  0.0      0     16  -  DL   30May24     0:00.25 [aiod4]
root      118   0.0  0.1  64512  17784  -  S    18:52       0:00.28 /usr/local/bin/php-cgi
root      150   0.0  0.1  59820  23304  -  I    15:05       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      173   0.0  0.1  59820  23368  -  I    15:41       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      216   0.0  0.0  27216      8  -  IWs  -           0:00.00 /usr/local/bin/python3 /usr/local/opnsense/service/configd.py (python3.11)
root      252   0.0  0.1  59820  23296  -  I    13:29       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      359   0.0  0.1  59820  23504  -  I    17:13       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      366   0.0  0.1  59820  23316  -  I    12:25       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      408   0.0  0.1  59820  23296  -  I    12:47       0:00.17 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      437   0.0  0.1  59820  23308  -  I    15:10       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      593   0.0  0.1  59820  23292  -  I    14:06       0:00.20 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      657   0.0  0.0  11552    724  -  Ss   30May24     0:03.14 /sbin/devd
root      670   0.0  0.2  59820  26572  -  I    18:52       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      689   0.0  0.2  59820  26560  -  I    18:54       0:00.16 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      691   0.0  0.1  59820  11948  -  I    10:42       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      730   0.0  0.1  59820  11952  -  I    09:44       0:00.18 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      734   0.0  0.1  59820  23300  -  I    14:47       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      752   0.0  0.1  59820  23468  -  I    12:20       0:00.18 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      753   0.0  0.1  53676  23664  -  I    18:08       0:00.17 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      770   0.0  0.1  59820  23476  -  I    12:00       0:00.17 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      849   0.0  0.1  59820  23316  -  I    15:08       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root      906   0.0  0.1  59820  23308  -  I    12:12       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force

Omitted in the middle

root    98843   0.0  0.1  59820  11944  -  I    10:57       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    98957   0.0  0.1  59820  23308  -  I    11:58       0:00.18 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    98967   0.0  0.1  59820  23284  -  I    14:40       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    98995   0.0  0.1  59820  11944  -  I    09:50       0:00.18 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99044   0.0  0.1  59820  23292  -  I    14:28       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99085   0.0  0.1  59820  23496  -  I    16:58       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99113   0.0  0.1  59820  23600  -  I    16:00       0:00.20 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99117   0.0  0.1  59820  23312  -  I    13:07       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99124   0.0  0.1  59820  11944  -  I    10:48       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99149   0.0  0.1  59820  23284  -  I    12:29       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99221   0.0  0.1  59820  23308  -  I    14:11       0:00.20 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99371   0.0  0.1  59820  23308  -  I    13:03       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99399   0.0  0.1  59820  11944  -  I    11:04       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99577   0.0  0.1  59820  23444  -  I    14:12       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99583   0.0  0.1  59820  11940  -  I    10:20       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99613   0.0  0.1  59820  11948  -  I    10:54       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99628   0.0  0.1  59820  23292  -  I    14:59       0:00.20 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99736   0.0  0.1  59820  11952  -  I    11:22       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99759   0.0  0.1  59820  23288  -  I    13:31       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99858   0.0  0.1  59820  11960  -  I    09:58       0:00.18 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99859   0.0  0.1  59820  23292  -  I    15:33       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99884   0.0  0.1  59820  11952  -  I    11:11       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    99890   0.0  0.1  59820  23692  -  I    18:18       0:00.19 /usr/local/bin/php /usr/local/etc/rc.newwanipv6 re0 force
root    54187   0.0  0.0  12744   1212 v0  Is+  30May24     0:00.00 /usr/libexec/getty Pc ttyv0
root    54263   0.0  0.0  12744   1208 v1  Is+  30May24     0:00.00 /usr/libexec/getty Pc ttyv1
root    54991   0.0  0.0  12744   1212 v2  Is+  30May24     0:00.00 /usr/libexec/getty Pc ttyv2
root    55328   0.0  0.0  12744   1212 v3  Is+  30May24     0:00.00 /usr/libexec/getty Pc ttyv3
root    55841   0.0  0.0  12744   1208 v4  Is+  30May24     0:00.00 /usr/libexec/getty Pc ttyv4
root    56466   0.0  0.0  12744   1212 v5  Is+  30May24     0:00.00 /usr/libexec/getty Pc ttyv5
root    56962   0.0  0.0  12744   1208 v6  Is+  30May24     0:00.00 /usr/libexec/getty Pc ttyv6
root    57204   0.0  0.0  12744   1212 v7  Is+  30May24     0:00.00 /usr/libexec/getty Pc ttyv7
root    12238   0.0  0.0  15900   4320  0  R+   18:58       0:00.01 ps -aux
root    35177   0.0  0.0  13784   3360  0  S    18:39       0:00.03 /bin/csh
root    94189   0.0  0.0  13488   2232  0  Is   18:39       0:00.01 /bin/sh /usr/local/sbin/opnsense-shell

ps -aux | grep rc.newwanipv6 | wc -l

root@OPNsense:~ # ps -aux |grep rc.newwanipv6 |wc -l
    1499

To Reproduce

Steps to reproduce the behavior:

Enabling WAN6 and waiting a while should reproduce it.

Expected behavior

Processes should exit or should not start processes indefinitely.

Screenshots

debug

Environment

Software version used and hardware type if relevant, e.g.:

Dell Optiplex 3070 MFF OPNsense 24.1.8-amd64 Intel(R) Core(TM) i3-8100T CPU @ 3.10GHz (4 cores, 4 threads) Network Intel I210 1 and Realtek NIC 1

fichtner commented 1 week ago

Try to avoid the Realtek interface(s) as WAN interface and it should disappear.

wevsty commented 1 week ago

Try to avoid the Realtek interface(s) as WAN interface and it should disappear.

When the problem occurs, I am using the realtek network card and the vendor driver as the WAN. I'm not sure if this is a realtek driver issue, but I understand that if it is a realtek driver issue then it is likely unsolvable.

I configured and tried to use the Intel NIC as the WAN interface and it worked without any problems for the past few hours. I close this issue for now. If the problem still occurs with Intel NIC in the future, I will reopen it.

fichtner commented 1 week ago

I'm not sure if this is a realtek driver issue, but I understand that if it is a realtek driver issue then it is likely unsolvable.

Yes I think it is. There is also the "os-realtek-re" plugin with the Realtek vendor driver that can help in such cases, but that's not my favourite recommendation because it does not hold support for very old re(4) devices anymore, but if that one works for you it's certainly beneficial to install the plugin, but make sure the interface remains usable after reboot (or check compatibility with your chipset first).

wevsty commented 1 week ago

Yes I think it is. There is also the "os-realtek-re" plugin with the Realtek vendor driver that can help in such cases,

I can confirm that I was already using the driver provided by the "os-realtek-re" plugin at the time of this problem. The BSD default driver is one I've used before, but I think that one has other issues as well.

For me, it's frustrating that the realtek NIC isn't removable on the motherboard and without being able to use it properly and consistently under BSD, I'm left with no choice but to try a different OS or replace the device with a new one.

fichtner commented 1 week ago

I'm left with no choice but to try a different OS or replace the device with a new one

Completely understandable. Realtek on FreeBSD has always been problematic.

Cheers, Franco