threefoldfoundation / tfchain

Official implementation of the ThreeFold blockchain
https://explorer.threefoldtoken.com
Apache License 2.0
13 stars 7 forks source link

test BC nodes to establish root cause for instability #313

Closed ramezsaeed closed 2 years ago

ramezsaeed commented 5 years ago
zaibon commented 5 years ago

@muhamadazmy something to think about, maybe having multiple endpoint for connectivity test is not a bad idea.

muhamadazmy commented 5 years ago

@zaibon yup.

ramezsaeed commented 5 years ago

https://github.com/threefoldtech/0-core/pull/136

christopherhutton commented 5 years ago

do we know this is definitely the root cause of the problem on the BC nodes ? @robvanmieghem @zaibon

christopherhutton commented 5 years ago

until we know the actual root cause then how can we close this issue ? @robvanmieghem @LeeSmet @ramezsaeed pls advise if the root cause is known ?

sabrinasadik commented 5 years ago
  • can you please add a second ip/url to check internet connectivity to the monitoring specs? right now we only use google.com, but if website down our nodes are down also.
  • zaibon said that the monitoring algo uses google.com to proof that the node has internet connection. last week google.com was not reachable through various isps in europe. so all the nodes connected via those isps showed no internet connection. so thinking to have a fallback url in the monitoring algo.

We thought this was the solution to the bc nodes instability, but even tho this was done, the problem still remains. Nodes are still going offline. Need to find the reason for this so reopening issue.

sabrinasadik commented 5 years ago

Turns out that a lot of the people whose nodes are offline, had a power outtage or unplugged the node to put into another plug, and when they did that the node did not reboot automatically. And some even died because of the sudden power outtage.