search

stacks-network / stacks-core

The Stacks blockchain implementation
https://docs.stacks.co
GNU General Public License v3.0
3k stars 659 forks source link

Regtest + krypton node crash #3285

Closed zone117x closed 1 year ago

zone117x commented 1 year ago

Currently running the next/Stacks2.1 branch with a regtest node and it crashed after ~2000 blocks.

Using this config: https://github.com/zone117x/stacks-regtest-env/blob/6d83990b01da137ec268d7eebeef4a15e3bcc780/stacks-krypton-miner.toml

Here's the logs I have. Unfortunately did not have debug logging enabled. I'll enable that and update this issue with more logs if it happens again.

stacks-regtest-env-stacks-node-1  | INFO [1662566501.706080] [src/burnchains/bitcoin/indexer.rs:732] [main] New canonical Bitcoin chain found! New tip is 65ed4e211365eab3be0e9eb4118e25c39f26f0678032bff06b18f3abd2b7d642
stacks-regtest-env-stacks-node-1  | INFO [1662566501.748881] [src/burnchains/burnchain.rs:1295] [main] Syncing Bitcoin blocks: 100.0% (2014 to 2015 out of 2014)
stacks-regtest-env-stacks-node-1  | INFO [1662566501.762037] [src/chainstate/burn/db/sortdb.rs:3982] [chains-coordinator] ACCEPTED(2015) leader block commit 54590ad704837f657d181dd401873aa11e6e37a77805a1e82511506ad95bbcfa at 2015,1, apparent_sender: 13iMbKEbE91ZNtgJMzjYmQEurex2QoKxZN
stacks-regtest-env-stacks-node-1  | INFO [1662566501.859940] [testnet/stacks-node/src/run_loop/neon.rs:676] [main] Downloaded burnchain blocks up to height 2015; target height is 2016; next_sortition_height = 2015, sortition_db_height = 2014
stacks-regtest-env-stacks-node-1  | INFO [1662566501.862359] [testnet/stacks-node/src/neon_node.rs:2302] [main] Received burnchain block #2015 including block_commit_op (winning) - miEJtNKa3ASpA19v5ZhvbKTEieYjLpzCYT (186251707556826a5b459b8d4adb7f5544744e07883f2bc3e6b4e7986a94f755)
stacks-regtest-env-stacks-node-1  | INFO [1662566501.862600] [testnet/stacks-node/src/neon_node.rs:1103] [relayer] Won sortition! Mining reward will be received in 100 blocks (block #2013)
stacks-regtest-env-stacks-node-1  | INFO [1662566501.862619] [testnet/stacks-node/src/run_loop/neon.rs:768] [main] Synchronized full burnchain up to height 2015. Proceeding to mine blocks
stacks-regtest-env-stacks-node-1  | INFO [1662566502.245126] [src/net/http.rs:1635] [p2p] Handle HTTPRequest, verb: GET, peer_addr: 172.18.0.2:52380, path: /v2/info, query: 
stacks-regtest-env-stacks-node-1  | INFO [1662566502.252943] [testnet/stacks-node/src/neon_node.rs:605] [relayer] Will keep polling mempool for transactions to include in a microblock
stacks-regtest-env-stacks-node-1  | ERRO [1662566502.935831] [testnet/stacks-node/src/main.rs:58] [main] Process abort due to thread panic: panicked at 'interval 0 is not partial', src/burnchains/bitcoin/spv.rs:472:9
stacks-regtest-env-stacks-node-1  | ERRO [1662566503.017170] [testnet/stacks-node/src/main.rs:60] [main] Panic backtrace:    0: stacks_node::main::{{closure}}
stacks-regtest-env-stacks-node-1  |              at /stacks/testnet/stacks-node/src/main.rs:59:18
stacks-regtest-env-stacks-node-1  |    1: std::panicking::rust_panic_with_hook
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:702:17
stacks-regtest-env-stacks-node-1  |    2: std::panicking::begin_panic_handler::{{closure}}
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:588:13
stacks-regtest-env-stacks-node-1  |    3: std::sys_common::backtrace::__rust_end_short_backtrace
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/sys_common/backtrace.rs:138:18
stacks-regtest-env-stacks-node-1  |    4: rust_begin_unwind
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:584:5
stacks-regtest-env-stacks-node-1  |    5: core::panicking::panic_fmt
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panicking.rs:142:14
stacks-regtest-env-stacks-node-1  |    6: blockstack_lib::burnchains::bitcoin::spv::SpvClient::get_chain_work
stacks-regtest-env-stacks-node-1  |              at /stacks/src/burnchains/bitcoin/spv.rs:472:9
stacks-regtest-env-stacks-node-1  |    7: blockstack_lib::burnchains::bitcoin::indexer::BitcoinIndexer::inner_find_bitcoin_reorg
stacks-regtest-env-stacks-node-1  |              at /stacks/src/burnchains/bitcoin/indexer.rs:721:35
stacks-regtest-env-stacks-node-1  |       blockstack_lib::burnchains::bitcoin::indexer::BitcoinIndexer::find_bitcoin_reorg
stacks-regtest-env-stacks-node-1  |              at /stacks/src/burnchains/bitcoin/indexer.rs:527:9
stacks-regtest-env-stacks-node-1  |       <blockstack_lib::burnchains::bitcoin::indexer::BitcoinIndexer as blockstack_lib::burnchains::indexer::BurnchainIndexer>::find_chain_reorg
stacks-regtest-env-stacks-node-1  |              at /stacks/src/burnchains/bitcoin/indexer.rs:968:9
stacks-regtest-env-stacks-node-1  |    8: blockstack_lib::burnchains::burnchain::<impl blockstack_lib::burnchains::Burnchain>::sync_reorg
stacks-regtest-env-stacks-node-1  |              at /stacks/src/burnchains/burnchain.rs:911:28
stacks-regtest-env-stacks-node-1  |    9: blockstack_lib::burnchains::burnchain::<impl blockstack_lib::burnchains::Burnchain>::sync_with_indexer
stacks-regtest-env-stacks-node-1  |              at /stacks/src/burnchains/burnchain.rs:1200:40
stacks-regtest-env-stacks-node-1  |   10: stacks_node::burnchains::bitcoin_regtest_controller::BitcoinRegtestController::receive_blocks
stacks-regtest-env-stacks-node-1  |              at /stacks/testnet/stacks-node/src/burnchains/bitcoin_regtest_controller.rs:403:19
stacks-regtest-env-stacks-node-1  |   11: <stacks_node::burnchains::bitcoin_regtest_controller::BitcoinRegtestController as stacks_node::burnchains::BurnchainController>::sync
stacks-regtest-env-stacks-node-1  |              at /stacks/testnet/stacks-node/src/burnchains/bitcoin_regtest_controller.rs:1569:13
stacks-regtest-env-stacks-node-1  |   12: stacks_node::run_loop::neon::RunLoop::start
stacks-regtest-env-stacks-node-1  |              at /stacks/testnet/stacks-node/src/run_loop/neon.rs:660:27
stacks-regtest-env-stacks-node-1  |   13: stacks_node::main
stacks-regtest-env-stacks-node-1  |              at /stacks/testnet/stacks-node/src/main.rs:209:9
stacks-regtest-env-stacks-node-1  |   14: core::ops::function::FnOnce::call_once
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5
stacks-regtest-env-stacks-node-1  |       std::sys_common::backtrace::__rust_begin_short_backtrace
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/sys_common/backtrace.rs:122:18
stacks-regtest-env-stacks-node-1  |   15: std::rt::lang_start::{{closure}}
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/rt.rs:145:18
stacks-regtest-env-stacks-node-1  |   16: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:280:13
stacks-regtest-env-stacks-node-1  |       std::panicking::try::do_call
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40
stacks-regtest-env-stacks-node-1  |       std::panicking::try
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19
stacks-regtest-env-stacks-node-1  |       std::panic::catch_unwind
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14
stacks-regtest-env-stacks-node-1  |       std::rt::lang_start_internal::{{closure}}
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/rt.rs:128:48
stacks-regtest-env-stacks-node-1  |       std::panicking::try::do_call
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40
stacks-regtest-env-stacks-node-1  |       std::panicking::try
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19
stacks-regtest-env-stacks-node-1  |       std::panic::catch_unwind
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14
stacks-regtest-env-stacks-node-1  |       std::rt::lang_start_internal
stacks-regtest-env-stacks-node-1  |              at /rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/rt.rs:128:20
stacks-regtest-env-stacks-node-1  |   17: main
stacks-regtest-env-stacks-node-1  |   18: __libc_start_main
stacks-regtest-env-stacks-node-1  |   19: _start
stacks-regtest-env-stacks-node-1  | 
stacks-regtest-env-stacks-node-1  | Dumping core for pid 1
jcnelson commented 1 year ago

Duplicated this in an integration test; working on a fix

zone117x commented 1 year ago

I can confirm that https://github.com/stacks-network/stacks-blockchain/pull/3286 fixes this issue for my test environment. Ty @jcnelson.