lichao-cobo
commented
4 months ago met the similar issue, when syncing, download header hanged about 10 mins. Then log shows banned a peer. │ 2024-06-17T06:28:56.019889Z WARN sync: Sync: ban a peer: ed25519:73iUsj3cLuU2tYgPj2wrycpNJNfkyACbT3rzGAJ1nuiS@65.21.22.25:24567, for not providing enough headers. Peer's height: 121342698 │
│ 2024-06-17T06:19:58.334767Z INFO stats: #121329433 Downloading blocks 12.02% (13264 left; at 121329433) 31 peers ⬇ 8.01 MB/s ⬆ 3.64 MB/s 0.01 bps 6.68 Tgas/s CPU: 113%, Mem: 10.2 GB │
│ 2024-06-17T06:20:16.041644Z INFO stats: #121329433 Downloading headers 99.66% (51 left; at 121342647) 31 peers ⬇ 8.01 MB/s ⬆ 3.64 MB/s 0.00 bps 0 gas/s CPU: 158%, Mem: 10.3 GB │
│ 2024-06-17T06:20:50.426400Z INFO stats: #121329433 Downloading headers 99.66% (51 left; at 121342647) 31 peers ⬇ 8.03 MB/s ⬆ 3.65 MB/s 0.00 bps 0 gas/s CPU: 154%, Mem: 10.3 GB │
│ 2024-06-17T06:28:56.019889Z WARN sync: Sync: ban a peer: ed25519:73iUsj3cLuU2tYgPj2wrycpNJNfkyACbT3rzGAJ1nuiS@65.21.22.25:24567, for not providing enough headers. Peer's height: 121342698 │
│ 2024-06-17T06:28:56.021724Z INFO near_network::peer_manager::connection: peer ed25519:73iUsj3cLuU2tYgPj2wrycpNJNfkyACbT3rzGAJ1nuiS disconnected, while sending SyncAccountsData │
│ 2024-06-17T06:28:56.021739Z INFO near_network::peer_manager::connection: peer ed25519:73iUsj3cLuU2tYgPj2wrycpNJNfkyACbT3rzGAJ1nuiS disconnected, while sending SyncAccountsData │
│ 2024-06-17T06:28:56.024367Z INFO stats: #121329434 Downloading headers 99.66% (51 left; at 121342647) 31 peers ⬇ 8.03 MB/s ⬆ 3.65 MB/s 0.00 bps 2.21 Tgas/s CPU: 96%, Mem: 10.4 GB │
│ 2024-06-17T06:29:00.181627Z WARN network: Banning peer ed25519:73iUsj3cLuU2tYgPj2wrycpNJNfkyACbT3rzGAJ1nuiS for ProvidedNotEnoughHeaders │
│ 2024-06-17T06:29:07.037009Z INFO stats: #121329435 Downloading blocks 11.89% (13445 left; at 121329435) 31 peers ⬇ 9.29 MB/s ⬆ 2.27 MB/s 0.09 bps 87.9 Tgas/s CPU: 366%, Mem: 12.1 GB │
│ 2024-06-17T06:29:17.020497Z INFO stats: #121329436 Downloading blocks 11.53% (13921 left; at 121329436) 30 peers ⬇ 43.2 MB/s ⬆ 3.31 MB/s 0.10 bps 83.4 Tgas/s CPU: 359%, Mem: 11.9 GB
I've observed a problem where a node will not make progress in header sync after it was restarted in the middle of header sync previously. It looks like this call is showing that the next header we need to apply is already known, when it hasn't actually marked as being the new header head, if a previous interrupt happened between committing a header and updating the header head to reflect that
If that's what's happening, this should be reproducible by inserting artificial interrupts between those two points and trying to see if header sync gets stuck