search

AstarNetwork / Astar

The dApp hub for blockchains of the future
https://astar.network/
GNU General Public License v3.0
739 stars 354 forks source link

'Digest item must match that calculated'-crash #288

Closed xni closed 3 years ago

xni commented 3 years ago

Hi! My plasm node is crashing sporadically

Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]: ====================
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]: Version: 1.9.0-3996f40-x86_64-linux-gnu
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:    0: sp_panic_handler::set::{{closure}}
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:    1: std::panicking::rust_panic_with_hook
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:              at /rustc/2987785df3d46d5ff144a5c67fbb8f5cca798d78/library/std/src/panicking.rs:595:17
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:    2: std::panicking::begin_panic::{{closure}}
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:    3: std::sys_common::backtrace::__rust_end_short_backtrace
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:    4: std::panicking::begin_panic
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:    5: frame_executive::Executive<System,Block,Context,UnsignedValidator,AllModules,COnRuntimeUpgrade>::final_checks
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:    6: tracing::span::Span::in_scope
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:    7: frame_executive::Executive<System,Block,Context,UnsignedValidator,AllModules,COnRuntimeUpgrade>::execute_block
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:    8: <plasm_runtime::Runtime as sp_api::runtime_decl_for_Core::Core<sp_runtime::generic::block::Block<sp_runtime::generic::header::Header<u32,sp_runtime::traits::BlakeTwo256>,sp_runtime::generic::unchecked_extrinsic::UncheckedExtrinsic<<pallet_indices::Module<plasm_runtime::Runtime> as sp_runtime::traits::StaticLookup>::Source,plasm_runtime::Call,sp_runtime::MultiSignature,(frame_system::extensions::check_spec_version::CheckSpecVersion<plasm_runtime::Runtime>,frame_system::extensions::check_tx_version::CheckTxVersion<plasm_runtime::Runtime>,frame_system::extensions::check_genesis::CheckGenesis<plasm_runtime::Runtime>,frame_system::extensions::check_mortality::CheckMortality<plasm_runtime::Runtime>,frame_system::extensions::check_nonce::CheckNonce<plasm_runtime::Runtime>,frame_system::extensions::check_weight::CheckWeight<plasm_runtime::Runtime>,pallet_transaction_payment::ChargeTransactionPayment<plasm_runtime::Runtime>)>>>>::execute_block
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:    9: sp_api::runtime_decl_for_Core::execute_block_native_call_generator::{{closure}}
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   10: std::thread::local::LocalKey<T>::with
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   11: sc_executor::native_executor::WasmExecutor::with_instance::{{closure}}
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   12: sc_executor::wasm_runtime::RuntimeCache::with_instance
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   13: <sc_executor::native_executor::NativeExecutor<D> as sp_core::traits::CodeExecutor>::call
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   14: sp_state_machine::execution::StateMachine<B,H,N,Exec>::execute_aux
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   15: sp_state_machine::execution::StateMachine<B,H,N,Exec>::execute_using_consensus_failure_handler
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   16: <sc_service::client::call_executor::LocalCallExecutor<B,E> as sc_client_api::call_executor::CallExecutor<Block>>::contextual_call
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   17: <sc_service::client::client::Client<B,E,Block,RA> as sp_api::CallApiAt<Block>>::call_api_at
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   18: sp_api::runtime_decl_for_Core::execute_block_call_api_at
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   19: sp_api::Core::execute_block_with_context
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   20: <&sc_service::client::client::Client<B,E,Block,RA> as sp_consensus::block_import::BlockImport<Block>>::import_block
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   21: <sc_finality_grandpa::import::GrandpaBlockImport<BE,Block,Client,SC> as sp_consensus::block_import::BlockImport<Block>>::import_block
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   22: <fc_consensus::FrontierBlockImport<B,I,C> as sp_consensus::block_import::BlockImport<B>>::import_block
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   23: <sc_consensus_babe::BabeBlockImport<Block,Client,Inner> as sp_consensus::block_import::BlockImport<Block>>::import_block
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   24: <alloc::boxed::Box<dyn sp_consensus::block_import::BlockImport<B>+Error = sp_consensus::error::Error+Transaction = Transaction+core::marker::Send+core::marker::Sync> as sp_consensus::block_import::BlockImport<B>>::import_block
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   25: sp_consensus::import_queue::import_single_block_metered
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   26: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   27: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   28: <sc_service::task_manager::prometheus_future::PrometheusFuture<T> as core::future::future::Future>::poll
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   29: <futures_util::future::select::Select<A,B> as core::future::future::Future>::poll
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   30: <tracing_futures::Instrumented<T> as core::future::future::Future>::poll
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   31: <sc_service::task_manager::WithTelemetrySpan<T> as core::future::future::Future>::poll
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   32: std::thread::local::LocalKey<T>::with
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   33: futures_executor::local_pool::block_on
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   34: tokio::runtime::task::core::Core<T,S>::poll
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   35: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   36: tokio::runtime::task::harness::Harness<T,S>::poll
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   37: tokio::runtime::blocking::pool::Inner::run
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   38: tokio::runtime::context::enter
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   39: std::sys_common::backtrace::__rust_begin_short_backtrace
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   40: core::ops::function::FnOnce::call_once{{vtable.shim}}
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   41: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:              at /rustc/2987785df3d46d5ff144a5c67fbb8f5cca798d78/library/alloc/src/boxed.rs:1328:9
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:       <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:              at /rustc/2987785df3d46d5ff144a5c67fbb8f5cca798d78/library/alloc/src/boxed.rs:1328:9
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:       std::sys::unix::thread::Thread::new::thread_start
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:              at /rustc/2987785df3d46d5ff144a5c67fbb8f5cca798d78/library/std/src/sys/unix/thread.rs:71:17
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   42: start_thread
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:   43: clone
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]: Thread 'tokio-runtime-worker' panicked at 'Digest item must match that calculated.', /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/frame-executive-3.0.0/src/lib.rs:399
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]: This is a bug. Please report it at:
Apr 15 23:38:38 Ubuntu-2010-groovy-64-minimal plasm[1474]:         https://github.com/PlasmNetwork/Plasm/issues/new
Apr 15 23:38:41 Ubuntu-2010-groovy-64-minimal plasm[1474]: 2021-04-15 23:38:41  ⚙️  Syncing 702.8 bps, target=#273447 (31 peers), best: #227514 (0xb6d0…fb60), finalized #227328 (0xd948…9265), ⬇ 200.3kiB/s ⬆ 2.5kiB/s
Apr 15 23:38:46 Ubuntu-2010-groovy-64-minimal plasm[1474]: 2021-04-15 23:38:46  ⚙️  Syncing 726.4 bps, target=#273447 (31 peers), best: #231146 (0xac09…a2f3), finalized #231084 (0x9bc6…e11d), ⬇ 268.0kiB/s ⬆ 2.5kiB/s
Apr 15 23:38:51 Ubuntu-2010-groovy-64-minimal plasm[1474]: 2021-04-15 23:38:51  ⚙️  Syncing 662.4 bps, target=#273448 (31 peers), best: #234458 (0x6382…9a5b), finalized #233984 (0x46b9…2672), ⬇ 268.8kiB/s ⬆ 2.0kiB/s

What else can I provide to help with investigation?

command: /home/plasm/plasm --validator --rpc-cors all --name 'validators_support'

$ /home/plasm/plasm --version
plasm 1.9.0-3996f40-x86_64-linux-gnu

HW: Intel(R) Xeon(R) CPU E3-1275 v6 @ 3.80GHz, 64 Gb, Ubuntu 20.04

fiexer commented 3 years ago

We are aware of this issue. This is a frontier issue, if your node is synced and running you can ignore this.