Closed AltCoiFish closed 2 years ago
Probably related to #2433
Agreeing with @friedger. The offending line is in the block-commit construction: https://github.com/blockstack/stacks-blockchain/blob/master/testnet/stacks-node/src/node.rs#L835. This unreachable!()
could have been reached if the sortition DB is corrupted -- namely, there exists a row in the snapshots
table where a sortition happened, but no corresponding block-commit could be found in the snapshot_transition_ops
table.
This shouldn't happen in develop
, because unlike master
, develop
has a safe shutdown procedure. @AltCoiFish Can you try using the develop
branch for the time being, if you need to restart your node frequently?
This shouldn't happen in
develop
, because unlikemaster
,develop
has a safe shutdown procedure. @AltCoiFish Can you try using thedevelop
branch for the time being, if you need to restart your node frequently?
Sure I can, as I want to try all features including mining mainnet etc , I have to restart the node from time to time. Please let me know in here when I can switch back.
Sure thing. We announce releases via announce@stacks.org
if you want to subscribe (link: https://groups.google.com/u/1/a/stacks.org/g/announce). When the fix makes it into master
, we'll make an announcement via that mailing list.
Its enough when you update to latest
in Github I am monitoring the latest
releases.
Feel free to close it once its fixed, or latest after I did test it.
FWIW, I also see this in develop. I start stacks-node and wait for it to settle, then send it SIGTERM, and re-run.
I have to nuke run/mocknet (working_dir), to start successfully.
[node]
name = "helium-node"
rpc_bind = "127.0.0.1:20443"
working_dir = "/home/allday/src/run"
[burnchain]
chain = "bitcoin"
mode = "mocknet"
commit_anchor_block_within = 5000
$ git branch -vv
* develop 18243c1d7 Merge pull request #2502 from blockstack/fix/2483
master 59363a2dc [origin/master] Merge pull request #2524 from blockstack/hotfix/2518+2491
(gdb) run
Starting program: /home/allday/src/stacks-blockchain/target/debug/stacks-node start --config=run/Stacks_allday.toml
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
INFO [1617054984.812046] [testnet/stacks-node/src/main.rs:80] [main] stacks-node 0.1.0 (develop:18243c1d7, debug build, linux [x86_64])
INFO [1617054984.812131] [testnet/stacks-node/src/main.rs:125] [main] Loading config at path run/Stacks_allday.toml
INFO [1617054984.817779] [testnet/stacks-node/src/event_dispatcher.rs:632] [main] Registering event observer at: localhost:8080
INFO [1617054984.864492] [src/chainstate/burn/db/sortdb.rs:3428] [main] ACCEPTED(46) leader key register abef4306e28dc42e1a644856e499a4ed3d9579d3ebe2ac998850048a3800c35c at 46,1
BOOTSTRAP WITH []
DENY NEIGHBORS []
[New Thread 0x7ffff7b83700 (LWP 6976)]
INFO [1617054984.880147] [testnet/stacks-node/src/node.rs:513] [main] Start HTTP server on: 127.0.0.1:20443
INFO [1617054984.880203] [testnet/stacks-node/src/node.rs:514] [main] Start P2P server on: 0.0.0.0:20444
INFO [1617054984.886913] [testnet/stacks-node/src/tenure.rs:70] [main] Node starting new tenure with VRF d58698bcbbe76d23a0867ed6f3d85aa7f8e79a64c5519b91e41434981dafa1ce
WARN [1617054985.884727] [src/net/p2p.rs:4302] [ThreadId(2)] Atlas: updating attachment inventory failed Not connected to peer network
INFO [1617054990.024931] [src/chainstate/stacks/miner.rs:750] [main] Include tx, tx: 69fb2eb5005327bff14bbc726550071ad35c7c0f6a050ae9038728c7607ca2bf, payload: Coinbase, origin: ST1RH5BP5HTFX0S38YF7VRZAESJZ01Y54B6HQE12G
INFO [1617054990.060963] [src/chainstate/stacks/miner.rs:933] [main] Miner: mined anchored block c14bde48d6fb81daf5c8180d5a9d811eca09a4c574fdb313122b1849028c859b height 1 with 1 txs, parent block 0000000000000000000000000000000000000000000000000000000000000000, parent microblock 0000000000000000000000000000000000000000000000000000000000000000 (0), state root = 5fae9107b24fe00ff6dd7418c0f125c5532544b969c14791980f5ca781d3ddee
INFO [1617054990.066478] [testnet/stacks-node/src/tenure.rs:101] [main] Finish tenure: c14bde48d6fb81daf5c8180d5a9d811eca09a4c574fdb313122b1849028c859b
Process abort due to thread panic
0: stacks_node::main::{{closure}}
at stacks-blockchain/testnet/stacks-node/src/main.rs:57
1: std::panicking::rust_panic_with_hook
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/panicking.rs:573
2: std::panicking::begin_panic::{{closure}}
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/panicking.rs:498
3: std::sys_common::backtrace::__rust_end_short_backtrace
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/sys_common/backtrace.rs:153
4: std::panicking::begin_panic
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/panicking.rs:497
5: stacks_node::node::Node::generate_block_commit_op
at stacks-blockchain/testnet/stacks-node/src/node.rs:917
6: stacks_node::node::Node::commit_artifacts
at stacks-blockchain/testnet/stacks-node/src/node.rs:689
7: stacks_node::run_loop::helium::RunLoop::start
at stacks-blockchain/testnet/stacks-node/src/run_loop/helium.rs:109
8: stacks_node::main
at stacks-blockchain/testnet/stacks-node/src/main.rs:148
9: core::ops::function::FnOnce::call_once
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/core/src/ops/function.rs:227
10: std::sys_common::backtrace::__rust_begin_short_backtrace
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/sys_common/backtrace.rs:137
11: std::rt::lang_start::{{closure}}
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/rt.rs:66
12: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/core/src/ops/function.rs:259
std::panicking::try::do_call
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/panicking.rs:373
std::panicking::try
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/panicking.rs:337
std::panic::catch_unwind
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/panic.rs:379
std::rt::lang_start_internal
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/rt.rs:51
13: std::rt::lang_start
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/rt.rs:65
14: main
15: __libc_start_main
16: _start
Dumping core for pid 6974
(gdb) bt
#0 0x00007ffff7db755b in kill () at ../sysdeps/unix/syscall-template.S:78
#1 0x0000555555991d47 in stacks_node::main::{{closure}} ()
at testnet/stacks-node/src/main.rs:70
#2 0x000055555753f988 in std::panicking::rust_panic_with_hook (payload=...,
message=..., location=<optimized out>) at library/std/src/panicking.rs:573
#3 0x0000555555b958fc in std::panicking::begin_panic::{{closure}} ()
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/panicking.rs:498
#4 0x0000555555b95779 in std::sys_common::backtrace::__rust_end_short_backtrace (f=...)
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/sys_common/backtrace.rs:153
#5 0x0000555555b95827 in std::panicking::begin_panic (msg=...)
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/panicking.rs:497
#6 0x0000555555990c8d in stacks_node::node::Node::generate_block_commit_op (
self=0x7fffffffb240, block_header_hash=..., burn_fee=20000,
key=0x7ffffffe96b8, burnchain_tip=0x7fffffff05f0, vrf_seed=...)
at testnet/stacks-node/src/node.rs:917
#7 0x000055555598e95b in stacks_node::node::Node::commit_artifacts (
self=0x7fffffffb240, anchored_block_from_ongoing_tenure=0x7fffffff0438,
burnchain_tip=0x7fffffff05f0, burnchain_controller=0x7ffffffea218,
burn_fee=20000) at testnet/stacks-node/src/node.rs:689
#8 0x00005555559881cd in stacks_node::run_loop::helium::RunLoop::start (
self=0x7fffffffadd8, expected_num_rounds=0)
at testnet/stacks-node/src/run_loop/helium.rs:109
#9 0x000055555580eaf3 in stacks_node::main ()
at testnet/stacks-node/src/main.rs:148
(gdb) bt full
#0 0x00007ffff7db755b in kill () at ../sysdeps/unix/syscall-template.S:78
No locals.
#1 0x0000555555991d47 in stacks_node::main::{{closure}} ()
at testnet/stacks-node/src/main.rs:70
pid = 6974
bt = backtrace::capture::Backtrace {frames: alloc::vec::Vec<backtrace::capture::BacktraceFrame> {buf: alloc::raw_vec::RawVec<backtrace::capture::BacktraceFrame, alloc::alloc::Global> {ptr: core::ptr::unique::Unique<backtrace::capture::BacktraceFrame> {pointer: 0x555558666ce0, _marker: core::marker::PhantomData<backtrace::capture::BacktraceFrame>}, cap: 32, alloc: alloc::alloc::Global}, len: 22}, actual_start_index: 4}
#2 0x000055555753f988 in std::panicking::rust_panic_with_hook (payload=...,
message=..., location=<optimized out>) at library/std/src/panicking.rs:573
ptr = core::ops::function::*mut Fn<(&core::panic::PanicInfo)> {pointer: 0x1 <error: Cannot access memory at address 0x1>, vtable: <synthetic pointer>}
info = core::panic::PanicInfo {payload: core::any::&Any {pointer: 0x7ffffffe9030 "\022\344VWUU\000", vtable: 0x555558477de8}, message: core::option::Option<&core::fmt::Arguments>::Some(0x0), location: 0x55555846ca38}
panics = <optimized out>
#3 0x0000555555b958fc in std::panicking::begin_panic::{{closure}} ()
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/panicking.rs:498
msg = "internal error: entered unreachable code"
loc = 0x55555846ca38
#4 0x0000555555b95779 in std::sys_common::backtrace::__rust_end_short_backtrace (f=...)
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/sys_common/backtrace.rs:153
No locals.
#5 0x0000555555b95827 in std::panicking::begin_panic (msg=...)
at /build/rustc-n7HJ8w/rustc-1.47.0+dfsg1+llvm/library/std/src/panicking.rs:497
loc = 0x55555846ca38
#6 0x0000555555990c8d in stacks_node::node::Node::generate_block_commit_op (
self=0x7fffffffb240, block_header_hash=..., burn_fee=20000,
key=0x7ffffffe96b8, burnchain_tip=0x7fffffff05f0, vrf_seed=...)
at testnet/stacks-node/src/node.rs:917
No locals.
#7 0x000055555598e95b in stacks_node::node::Node::commit_artifacts (
self=0x7fffffffb240, anchored_block_from_ongoing_tenure=0x7fffffff0438,
burnchain_tip=0x7fffffff05f0, burnchain_controller=0x7ffffffea218,
burn_fee=20000) at testnet/stacks-node/src/node.rs:689
vrf_proof = blockstack_lib::util::vrf::VRFProof {Gamma: curve25519_dalek::edwards::EdwardsPoint {X: curve25519_dalek::backend::serial::u64::field::FieldElement51 ([1141553949753196, 953334841490310, 1430707968961643,
--Type <RET> for more, q to quit, c to continue without paging--
878844089311985,
202974272230747]), Y: curve25519_dalek::backend::serial::u64::field::FieldElement51 ([109902138737897, 1839481976554434, 1341110852017549,
1827406448531303,
973088160977663]), Z: curve25519_dalek::backend::serial::u64::field::FieldElement51 ([235143236321787, 1420495817862136, 464999991873051,
2111372066328871,
571446798623726]), T: curve25519_dalek::backend::serial::u64::field::FieldElement51 ([99445695996465, 2217413229295558, 42250458944784,
1491070346405972,
1613376548973058])}, c: curve25519_dalek::scalar::Scalar {bytes: [58, 126, 116, 226, 106, 222, 38, 235, 250, 188, 83, 214, 97, 24, 78, 222,
0 <repeats 16 times>]}, s: curve25519_dalek::scalar::Scalar {bytes: [55, 116, 82, 212, 208, 151, 37, 125, 211, 91, 186, 162, 40, 195, 105, 162,
74, 117, 169, 35, 177, 192, 251, 37, 93, 73, 123, 108, 193, 172,
235, 7]}}
registered_key = stacks_node::run_loop::RegisteredKey {block_height: 46, op_vtxindex: 1, vrf_public_key: blockstack_lib::util::vrf::VRFPublicKey (ed25519_dalek::public::PublicKey (curve25519_dalek::edwards::CompressedEdwardsY ([
226, 141, 250, 50, 88, 128, 209, 89, 111, 246, 32, 206, 228,
201, 237, 218, 250, 101, 38, 133, 34, 231, 212, 239, 103,
50, 194, 148, 190, 192, 189,
214]), curve25519_dalek::edwards::EdwardsPoint {X: curve25519_dalek::backend::serial::u64::field::FieldElement51 ([459106372340943,
1936129520762177, 248324102826894, 19428648008476,
1873826108731204]), Y: curve25519_dalek::backend::serial::u64::field::FieldElement51 ([1581838745873033, 152413486418133, 270093794180379,
1029925889466726,
2200186216955961]), Z: curve25519_dalek::backend::serial::u64::field::FieldElement51 ([1848282576875583, 1799833687170756,
480994356806904, 1733218614750612,
518221463986250]), T: curve25519_dalek::backend::serial::u64::field::FieldElement51 ([1640834644100035, 1274572672629180, 418528156212002,
1955254256117454, 2120612300149554])}))}
#8 0x00005555559881cd in stacks_node::run_loop::helium::RunLoop::start (
self=0x7fffffffadd8, expected_num_rounds=0)
at testnet/stacks-node/src/run_loop/helium.rs:109
artifacts_from_1st_tenure = stacks_node::tenure::TenureArtifacts {anchored_block: blockstack_lib::chainstate::stacks::StacksBlock {header: blockstack_lib::chainstate::stacks::StacksBlockHeader {version: 0, total_work: blockstack_lib::chainstate::stacks::StacksWorkScore {burn: 0, work: 1}, proof: blockstack_lib::util::vrf::VRFProof {Gamma: curve25519_dalek::edwards::EdwardsPoint {X: curve25519_dalek::backend::serial::u64::field::FieldElement51 ([1141553949753196,
953334841490310, 1430707968961643, 878844089311985,
202974272230747]), Y: curve25519_dalek::backend::serial::u--Type <RET> for more, q to quit, c to continue without paging--q
Ubuntu 20
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This issue has been automatically closed. Please reopen if needed.
This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Describe the bug I am running a
debian 10
64bit with current binaries:INFO [1616314766.501997] [testnet/stacks-node/src/main.rs:80] [main] stacks-node 2.0.10 (master:59363a2+, release build, linux [x86_64])
from https://github.com/blockstack/stacks-blockchain/releases/tag/2.0.10with following
mainnet-miner-conf.toml
config:starting parameter of the node:
/mnt/stx/node/stacks-node start --config /mnt/stx/node/mainnet-miner-conf.toml
I also did try the following
working_dir
syntax:working_dir = "/mnt/stx/data/"
(slash at the end)When starting the node, everything is working as expected. But I had to shutdown the node and re-start it and it
panicked
with the following error. If you delete the contents ofworking_dir
it's working again and you can start the node without any problems.If you need debug logs let me know.