search

near / stakewars-iii

Stake Wars: Episode 3 challenges and place to report issues
87 stars 177 forks source link

Node can not get enough peers after restarting #131

Closed mazito closed 2 years ago

mazito commented 2 years ago

After restarting the node, there seems to be impossible to now acquire the min 12 peers required for validation and so losing chunks.

My node (idtcn2) has been running with an average uptime of 94.4 over more than 90 epochs and was usually working with some 90 peers when in full regime.

After restarting with commit fe435d02c5ea497933c89d5e7d1703d9379b7e1f it has been at 0% uptime for more than two hours. Seems the the protocol change to 101 has serious problems at least for some nodes that were working quite welll (mine included, but is happening to many others) like: 

| stakewars-fi-1.factory.shardnet.near      | 6,640   | 1       | 28.57%   |               6 |               7 |               0 |              14 |

who was first in the scoreboard for the last 10 days.

Aug 28 01:21:41 idtcn3 neard[44332]: 2022-08-28T01:21:41.203291Z  INFO near_network::peer_manager::peer_manager_actor: Bandwidth stats total_bandwidth_used_by_all_peers=18742419 total_msg_received_count=28238 max_max_record_num_messages_in_progress=68 

Aug 28 01:21:46 idtcn3 neard[44332]: 2022-08-28T01:21:46.316128Z  INFO stats: # 2699829 C28uUZpd46yxLPm9L3cwstKvrdYhGzE6woy4BHAGfNS2 Validator | 100 validators 11 peers ⬇ 221 kB/s ⬆ 201 kB/s 0.40 bps 421 Ggas/s CPU: 140%, Mem: 2.98 GB 

Aug 28 01:21:56 idtcn3 neard[44332]: 2022-08-28T01:21:56.318197Z  INFO stats: # 2699835 48qvhNq8eQZsUunEAn3Pa7zQNup7sbBRPc1jGyMoWATN Validator | 100 validators 11 peers ⬇ 230 kB/s ⬆ 211 kB/s 0.60 bps 319 Ggas/s CPU: 69%, Mem: 2.98 GB 

Aug 28 01:22:06 idtcn3 neard[44332]: 2022-08-28T01:22:06.319827Z  INFO stats: # 2699843 NXR14AfQ3tEn2j169V4rC7tAp5TRFJQg63EKwxk4566 Validator | 100 validators 11 peers ⬇ 226 kB/s ⬆ 206 kB/s 0.70 bps 486 Ggas/s CPU: 64%, Mem: 2.98 GB 

Aug 28 01:22:16 idtcn3 neard[44332]: 2022-08-28T01:22:16.320327Z  INFO stats: # 2699850 EGAeVGjejmoJX7VsuvFi56ryzP6EfHDvPJKoCqhc7EYX Validator | 100 validators 11 peers ⬇ 229 kB/s ⬆ 206 kB/s 0.60 bps 90.3 Tgas/s CPU: 70%, Mem: 2.98 GB
mazito commented 2 years ago

UPDATE: After about two and a half hours it finally reached 20 peers and then began producing chunks again. It took some 30 lost chunks to get to this quantity of peers and start producing.

DDeAlmeida commented 2 years ago

ok, nice. Closing