search

tari-project / tari

The Tari protocol
https://tari.com
BSD 3-Clause "New" or "Revised" License
354 stars 219 forks source link

Mining nodes can leave gaps in database #1991

Closed CjS77 closed 3 years ago

CjS77 commented 4 years ago

Describe the bug If a node mines a block that is invalid, the DB can be left in an inconsistent state. Specifically, the header and chain metadata fields get updated as if the block were valid, whereas they should be cleaned up after validation failure.

Example: Running check-db listed several blocks as missing on my local node. The logs indicated that in each case, my node had mined a block at that height but was considered invalid.

Here's the log of the node trying to add the newly mined block (#31415):

2020-06-11 20:10:40.703260882 [c::m::blake_miner] DEBUG Miner found nonce: 11434140833061363953                                                                                        │
│2020-06-11 20:10:40.703332054 [c::m::miner] INFO  Mined a block: ----------------- Block -----------------                                                                             │
│--- Header ---                                                                                                                                                                         │
│Version: 1                                                                                                                                                                             │
│Block height: 31415                                                                                                                                                                    │
│Previous block hash: ed3eebb95614d896507344cf2f4d0095dceb4f4b2ea596944dea2a012b15b366                                                                                                  │
│Timestamp: Thu, 11 Jun 2020 18:10:35 +0000                                                                                                                                             │
│Merkle roots:                                                                                                                                                                          │
│Outputs: a126a2e252c4d5db2c916a4058fc656e14302225e5e1a1058ae2af3197978c3f                                                                                                              │
│Range proofs: f72b0d60e409e96daa7406f059eaaa0a2008cd5382debb1cfbe54f52f7e4afc5                                                                                                         │
│Kernels: 8923824d15a8beb31c301a9a35eae135657df7550de98a3dc4582868dfb0935d                                                                                                              │
│Total offset: 78d8461d8c35d9620eac3aba86b034c012b96cba09e45ee3ecef129d8634370c                                                                                                         │
│Nonce: 11434140833061363953                                                                                                                                                            │
│Proof of work:                                                                                                                                                                         │
│Mining algorithm: Blake, Target difficulty: 292509316                                                                                                                                  ┤
│Total accumulated difficulty:                                                                                                                                                          │
│Monero=1, Blake=165233848287113                                                                                                                                                        │
│Pow data:                                                                                                                                                                              │
│                                                                                                                                                                                       │
│---  Body  ---                                                                                                                                                                         ┤
│--- Transaction Kernels ---                                                                                                                                                            ┤
│Kernel 0:                                                                                                                                                                              ┤
│Fee: 750 µT                                                                                                                                                                            │
│Lock height: 0                                                                                                                                                                         │
│Features: (empty)                                                                                                                                                                      │
│Excess: 22e65472c1ca2cca06c3f06dcc5084990ed60415fdab30bc3cbdc2ae177a3557                                                                                                               │
│Excess signature: {"public_nonce":"3604eb9b06dcb791a2d4a5b5187b844b916ea6b2b699ade4c116b5e29d376003","signature":"95f65f119c00e16ea05e9eb6104583bcf46b0e5af349723022001765a1f0f808"}   │
│Meta_info: None                                                                                                                                                                        │
│Linked_kernel: None                                                                                                                                                                    │
│                                                                                                                                                                                       │
│Kernel 1:                                                                                                                                                                              │
│Fee: 0 µT                                                                                                                                                                              │
│Lock height: 0                                                                                                                                                                         │
│Features: COINBASE_KERNEL                                                                                                                                                              │
│Excess: 1619807ddc7b29611e08ebd9fac4c3cb5d97bc00cbfe30be96f0ca7b39132c0d                                                                                                               │
│Excess signature: {"public_nonce":"16ff3aefb7486a20d9c3d89b7b5e4d4c540f429162c987d81367f01a5b39bc16","signature":"fcacf59a315df65b09dc78bcfd5124eb046b0405fc6a1fe44a0fd47642d1ca01"}   │
│Meta_info: None       
Linked_kernel: None                                                                                                                                                                    │
│                                                                                                                                                                                       │
│--- Inputs (1) ---                                                                                                                                                                     ┤
│46b6f5fbb9b11a57588a5b971688d8d9d9d41c0727433e7a38eefa078e02966b [OutputFeatures { flags: COINBASE_OUTPUT, maturity: 31395 }]                                                          │
│--- Outputs (3) ---                                                                                                                                                                    │
│b65f46503fdd5751aa01411a818d8a49966eb09506e87a4cc8428e7ab1c7b030 [OutputFeatures { flags: (empty), maturity: 0 }] Proof: 00aead22c1da367a..95eff28c56bfa208                            │
│f66f24eb62e9a69bfb50e06db9ca4886530f9ffcf511793b612b3ff597fb785b [OutputFeatures { flags: (empty), maturity: 0 }] Proof: 52cd0f3d12697c90..f6b43172eb1fef09                            │
│3050d1f564fbdae3f27b8375cba4c2068a1939d126bec002b9328180f75b7633 [OutputFeatures { flags: COINBASE_OUTPUT, maturity: 31475 }] Proof: 5afb5e109b8ac304..b132c1671fd89509                │
│                                                                                                                                                                                       │
│                                                                                                                                                                                       │
│2020-06-11 20:10:40.703569810 [c::bn::comms_interface::inbound_handler] DEBUG Block #31415 (4f87cfac6533c43ce8dd7939da15cdd20cefbf74662ccd5fb274033880810ea3) received from local servi│
│2020-06-11 20:10:40.703655042 [c::val::block_validators] DEBUG block #31415 (4f87cfac6533c43ce8dd7939da15cdd20cefbf74662ccd5fb274033880810ea3) has PASSED stateless VALIDATION check.  │
│2020-06-11 20:10:40.722423230 [c::cs::database] DEBUG Added candidate block #31415 (4f87cfac6533c43ce8dd7939da15cdd20cefbf74662ccd5fb274033880810ea3) to the orphan database. Best heig│
│2020-06-11 20:10:40.722444767 [c::cs::database] DEBUG Checking if block #31415 (4f87cfac6533c43ce8dd7939da15cdd20cefbf74662ccd5fb274033880810ea3) is connected to the main chain.      │
│2020-06-11 20:10:40.722467688 [c::cs::database] DEBUG Connection with main chain found at block #31414 (ed3eebb95614d896507344cf2f4d0095dceb4f4b2ea596944dea2a012b15b366) from block #3│
│2020-06-11 20:10:40.760798313 [c::cs::database] DEBUG Comparing candidate block #31415 (accum_diff:12854347, hash:4f87cfac6533c43ce8dd7939da15cdd20cefbf74662ccd5fb274033880810ea3) to │
│2020-06-11 20:10:40.760910979 [c::cs::database] DEBUG Validate and add 1 chain blocks from height 31414.                                                                               │
│2020-06-11 20:10:40.854198830 [c::val::block_validators] DEBUG Block validation: Block is VALID for block #31415 (4f87cfac6533c43ce8dd7939da15cdd20cefbf74662ccd5fb274033880810ea3)    ┤
│2020-06-11 20:10:41.473355720 [c::bn::comms_interface::inbound_handler] WARN  Block validation failed: UnspendableInput                                                                │
│2020-06-11 20:10:41.473433558 [c::m::miner] ERROR Miner submitted invalid block. UnspendableInput.                                                                                     │
│2020-06-11 20:10:41.506748179 [c::bn::initialization] INFO  🤑💰🤑 Newly mined coinbase output added to wallet 🤑💰🤑                                                                  │
│2020-06-11 20:10:47.399139616 [c::bn::chain_state_sync_service] DEBUG Received chain metadata from NodeId '3ff68ac4d23e
...
2020-06-11 20:11:24.459096055 [c::bn::comms_interface::inbound_handler] DEBUG Block #31415 (7fbbc9cc0bc5b155539e9d12766a19472aa0ab10f68b66d1055498f288477ea8) received from remote peer│
│2020-06-11 20:11:24.459219292 [c::val::block_validators] DEBUG block #31415 (7fbbc9cc0bc5b155539e9d12766a19472aa0ab10f68b66d1055498f288477ea8) has PASSED stateless VALIDATION check.  ┤
│2020-06-11 20:11:24.472864028 [c::cs::database] DEBUG Added candidate block #31415 (7fbbc9cc0bc5b155539e9d12766a19472aa0ab10f68b66d1055498f288477ea8) to the orphan database. Best heig│
│2020-06-11 20:11:24.472887651 [c::cs::database] DEBUG Checking if block #31415 (7fbbc9cc0bc5b155539e9d12766a19472aa0ab10f68b66d1055498f288477ea8) is connected to the main chain.      │
│2020-06-11 20:11:24.472910457 [c::cs::database] DEBUG Connection with main chain found at block #31414 (ed3eebb95614d896507344cf2f4d0095dceb4f4b2ea596944dea2a012b15b366) from block #3│
│2020-06-11 20:11:24.510534734 [c::cs::database] DEBUG Comparing candidate block #31415 (accum_diff:12854344, hash:7fbbc9cc0bc5b155539e9d12766a19472aa0ab10f68b66d1055498f288477ea8) to │
│2020-06-11 20:11:24.510590409 [c::cs::database] DEBUG Validate and add 1 chain blocks from height 31414.                                                                               │
│2020-06-11 20:11:24.602434074 [c::val::block_validators] DEBUG Block validation: Block is VALID for block #31415 (7fbbc9cc0bc5b155539e9d12766a19472aa0ab10f68b66d1055498f288477ea8)    │
│2020-06-11 20:11:24.631676863 [c::cs::database] DEBUG Removing orphan blocks used for reorg.

However block 31415 is not in the db:

>> get-block 31415
Block not found at height 31415
│2020-06-12 09:46:46.062994040 [c::bn::comms_interface::inbound_handler] DEBUG Could not provide requested block 31415 to peer because: The requested value 'Spent output (795143a558939┤

The header is in the DB though:

Header hash: 7fbbc9cc0bc5b155539e9d12766a19472aa0ab10f68b66d1055498f288477ea8
Version: 1
Block height: 31415
Previous block hash: ed3eebb95614d896507344cf2f4d0095dceb4f4b2ea596944dea2a012b15b366
Timestamp: Thu, 11 Jun 2020 18:10:56 +0000
Merkle roots:
Outputs: e735a800bb638dd151983a34b5ba84bc0ada8b7310b9fce593ca205cbbd38ad4
Range proofs: 566ef75fe4c544a5a780b7c72115a72a15f0dfda2962931b7fee627b236a6b54
Kernels: 2dee9df05efe883e5480372fd44b6193396649aeea1a999cc3b393825ae70d7c
Total offset: 0000000000000000000000000000000000000000000000000000000000000000
Nonce: 17574085853965233666
Proof of work:
Mining algorithm: Blake, Target difficulty: 292509316
Total accumulated difficulty:
Monero=1, Blake=165233848287113
Pow data:

To Reproduce This error is not consistently reproducible.

Desktop (please complete the following information):

stringhandler commented 3 years ago

I think the new DB structure fixes this. Can reopen if that's not the case