Umbrel suddenly starts to crash after running one week without issues

[marviins87]

My Umbrel node started crashing every few hours since a few days. It was running without issues one week and it suddenly started. I'm unable to find the cause of it. It's a Pi 4 (8GB) with a SanDisk microSD card + 1TB Samsung SSD portable T5 disk and it's powered by the original Pi 3A power supply.

• Power

  vcgencmd get_throttled
  throttled=0x0

• I did a fcsk on both the SD and SSD but no errors were found.

• I'm able to boot the unit but after a few hours it's not replying on the NIC anymore so I need to disconnect the power.

• ./debug --upload says:

This script could not automatically detect an issue with your Umbrel. Please share the following links and paste it in the Umbrel Telegram group (https://t.me/getumbrel) so we can help you with your problem. https://umbrel-paste.vercel.app/171319d544c968b72d5d0110ece367e2

Anyone knows what is happening with my node or any other troubleshooting steps I should try?

[bolaum]

Maybe it's related to a problem I was having recently. In my case, it wasn't really crashing but network buffers getting full. Not even ssh was working after a few hours, but the board didn't seem to crash. Investigating logs, I found some weird messages regarding conntrack. I did the following:

echo 'nf_conntrack' >> /etc/modules

Add the following to the end of /etc/sysctl.d/99-sysctl.conf:

net.netfilter.nf_conntrack_max = 1048576
net.netfilter.nf_conntrack_tcp_timeout_established = 600
net.netfilter.nf_conntrack_generic_timeout = 60

Reboot.

Check new values after reboot with:

sysctl net.netfilter.nf_conntrack_max
sysctl net.netfilter.nf_conntrack_tcp_timeout_established
sysctl net.netfilter.nf_conntrack_generic_timeout

This will load nf_conntrack module and increase the max number of connections. It will also decrease the time for connections timeout.

After these changes, it seems stable. Hope it helps.

@louneskmt @AaronDewes do you think this deserves a PR?

[AaronDewes]

There is another network related issue which is more likely to happen, yours seems to happen very rarely

[jonsyu]

Closing due to inactivity