Open Vahe1994 opened 1 year ago
Looks like some issue with long-term circuit relays -- OR it could be caused by temporarily severed internet connection on your side.
If latter is the case, a typical failure scenario looks like this:
[based on a quick chat with @Vahe1994 ]
Suggested solution: during the next sprint (T + 3-4 weeks), when we work on means to run Petals server in background, we can add a check that the server is present on the health.petals.ml dashboard. If it isn't (but the dashboard is up), restart the server.
Server under the NAT stop being detected by others after several days working fine. The server stopped being shown in health.petals and requests to it stopped coming. There isn't any errors or logs that indicates that something went wrong. Last logs from the server are attached below![image](https://user-images.githubusercontent.com/4429900/217261476-346fbe4b-54d9-4346-a730-f04c4d0e04a1.png)