Header not found, Parent not found, Stage 5 verification failed

[SurfingNerd]

example:

2023-09-08 08:28:18  Header not found? : retract_step 2 n: 2793 h: 0x50cb…bd0a last_imported_block: 2794 last_imported_hash: 0xd41b…55b7 oldest_reorg 2719
2023-09-08 08:28:19  Stage 5 verification failed for #2794 (0xc72a…187c)
Block is ancient (current best block: #2794).
2023-09-08 08:28:19  Block import failed for #2795 (0xeb42…c312): Parent not found (0xc72a…187c) 
2023-09-08 08:28:19  
Bad block detected: Error(Msg("Parent not found"), State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
RLP: f90254f9024fa0c72a9de932fe45fd6841d1b1694da058fdd8f7b096d12191bc171748b0aa187ca01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347940000000000000000000000000000000000000000a0e0ebb9f839743d6066ba5e54f69185bac0870938ce9945d91a97ff33145886dea056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421a056e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421b901000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001820aeb8411e1a300808464fb192ca09930f25f236a98804bad8cea85ef68979759f64109928f666ee379fd27f0a089b860b9e39c3e44128b1b1329efa6af0d2feb307d88727b17e4a4ec96827a474b45ef6a5dcfe0fb19f90079d996222443be0c0c2c73712ed5e3bfff35eadab0877c6275a81b40ed154a99cc31151a408b642cc7f0819b5b7d7328bab4ed8110f4474cc0c0
Header: Header { parent_hash: 0xc72a9de932fe45fd6841d1b1694da058fdd8f7b096d12191bc171748b0aa187c, timestamp: 1694177580, number: 2795, author: 0x0000000000000000000000000000000000000000, transactions_root: 0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421, uncles_hash: 0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347, extra_data: [153, 48, 242, 95, 35, 106, 152, 128, 75, 173, 140, 234, 133, 239, 104, 151, 151, 89, 246, 65, 9, 146, 143, 102, 110, 227, 121, 253, 39, 240, 160, 137], state_root: 0xe0ebb9f839743d6066ba5e54f69185bac0870938ce9945d91a97ff33145886de, receipts_root: 0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421, log_bloom: 0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000, gas_used: 0, gas_limit: 300000000, difficulty: 1, seal: [[184, 96, 185, 227, 156, 62, 68, 18, 139, 27, 19, 41, 239, 166, 175, 13, 47, 235, 48, 125, 136, 114, 123, 23, 228, 164, 236, 150, 130, 122, 71, 75, 69, 239, 106, 93, 207, 224, 251, 25, 249, 0, 121, 217, 150, 34, 36, 67, 190, 12, 12, 44, 115, 113, 46, 213, 227, 191, 255, 53, 234, 218, 176, 135, 124, 98, 117, 168, 27, 64, 237, 21, 74, 153, 204, 49, 21, 26, 64, 139, 100, 44, 199, 240, 129, 155, 91, 125, 115, 40, 186, 180, 237, 129, 16, 244, 71, 76]], base_fee_per_gas: None, hash: Some(0xeb42fe464afb0f8b342f0879ac964ca2ffa765b0f6d18659041450987d17c312) }
Uncles: 
Transactions:

2023-09-08 08:28:21  Syncing    #2794 0xd41b…55b7     0.00 blk/s    0.0 tx/s    0.0 Mgas/s      0+    0 Qed LI:#2920    9/27 peers   132 KiB chain 0 bytes queue  RPC:  0 conn,    0 req/s,    0 µs
2023-09-08 08:28:26  Syncing
``

[SurfingNerd]

alpha 2 network: if a node is stuck, it is usually at the same block. Those blocks are:

• 3538
• 3436
• 2794

are they the first block of an epoch ??

[SurfingNerd]

does it only happen with "more up to date" blocks ?

[SurfingNerd]

the logic on the blockchain changed with block: 2791 - the miner contract information got included into the Block. other block number where the behavior switched: 3275

[SurfingNerd]

The problem in this case was connected that Nodes with the wrong node version took over the validator role.

[SurfingNerd]

Is it connected to Nodes that loose the active validator status ?