althea-net / rita

Rita is a routing and billing protocol that allows devices to buy and sell bandwidth
https://docs.althea.net/
Apache License 2.0
90 stars 24 forks source link

Overhaul automated rescue options #849

Closed jkilpatr closed 9 months ago

jkilpatr commented 9 months ago

This patch represents a complete overhaul of the automated rescue systems in rita

1) everywhere where we used to try and crash actix to force a restart, we instead do a system reboot. Logs show these rescue functions are invoked pretty often, but killing the actix thread does not actually stop Rita like was originally hoped leaving routers in a bad state

2) The ping check that's used to reboot client routers when they can't reach the internet has been overhauld to ping a lot more destinations and combined with the hAP overload check

For (1) I've been looking for the reason routers would simply stop opening tunnels and operating correctly. Reviewing this broken loop rescue behavior seems like a smoking gun for that sort of bug.