CitizenSearcher
commented
1 month ago After some experimentation, I found when compiling without RUSTFLAGS="-C target-cpu=native", the segfaults stop.
Open CitizenSearcher opened 1 month ago
CitizenSearcher
commented
1 month ago After some experimentation, I found when compiling without RUSTFLAGS="-C target-cpu=native", the segfaults stop.
dallonby
commented
1 month ago same issue, same solution. I'm also running on AMD CPU.
eierina
commented
3 weeks ago Few details on this issue from my side:
encode_fields in crates/primitives/src/transaction/legacy.rs which was modified in v1.0.2)rust-lldb /mnt/eth/store/reth-src/target/maxperf-debug/deps/reth-ac3b809190595c73 /tmp/core-tokio-runtime-w-11-1000-1000-202748-1724080468
(lldb) bt
* thread #1, name = 'reth', stop reason = signal SIGSEGV: invalid address
* frame #0: 0x00007fec1eb8000d libc.so.6`__memmove_avx512_unaligned_erms + 13
frame #1: 0x0000562207b4c67d reth-ac3b809190595c73`_$LT$alloc..vec..Vec$LT$u8$GT$$u20$as$u20$bytes..buf..buf_mut..BufMut$GT$::put_slice::hebe8c90fc75baf6a [inlined] core::intrinsics::copy_nonoverlapping::h6b441d860ac4e652 at intrinsics.rs:2959:14
frame #2: 0x0000562207b4c677 reth-ac3b809190595c73`_$LT$alloc..vec..Vec$LT$u8$GT$$u20$as$u20$bytes..buf..buf_mut..BufMut$GT$::put_slice::hebe8c90fc75baf6a [inlined] alloc::vec::Vec$LT$T$C$A$GT$::append_elements::h4cb784e1421b060c(self=0x00007fec0e3fd7d8, other=*const [u8] @ 0x00000000369e0ff0) at mod.rs:2147:18
frame #3: 0x0000562207b4c658 reth-ac3b809190595c73`_$LT$alloc..vec..Vec$LT$u8$GT$$u20$as$u20$bytes..buf..buf_mut..BufMut$GT$::put_slice::hebe8c90fc75baf6a [inlined] _$LT$alloc..vec..Vec$LT$T$C$A$GT$$u20$as$u20$alloc..vec..spec_extend..SpecExtend$LT$$RF$T$C$core..slice..iter..Iter$LT$T$GT$$GT$$GT$::spec_extend::h261c12bfbd9ed9be(self=0x00007fec0e3fd7d8, iterator=<unavailable>) at spec_extend.rs:55:18
frame #4: 0x0000562207b4c658 reth-ac3b809190595c73`_$LT$alloc..vec..Vec$LT$u8$GT$$u20$as$u20$bytes..buf..buf_mut..BufMut$GT$::put_slice::hebe8c90fc75baf6a [inlined] alloc::vec::Vec$LT$T$C$A$GT$::extend_from_slice::h4fa86c5faa78e1bf(self=0x00007fec0e3fd7d8, other=size=1060) at mod.rs:2591:9
frame #5: 0x0000562207b4c658 reth-ac3b809190595c73`_$LT$alloc..vec..Vec$LT$u8$GT$$u20$as$u20$bytes..buf..buf_mut..BufMut$GT$::put_slice::hebe8c90fc75baf6a(self=0x00007fec0e3fd7d8, src=size=1060) at buf_mut.rs:1628:9
frame #6: 0x000056220827b0ee reth-ac3b809190595c73`reth_primitives::transaction::legacy::TxLegacy::encode_fields::h4467f863b49f5f72 [inlined] _$LT$$u5b$u8$u5d$$u20$as$u20$alloy_rlp..encode..Encodable$GT$::encode::h92e325f31777a58d(self=size=1060, out=&mut dyn bytes::buf::buf_mut::BufMut @ 0x00000000380373b0) at encode.rs:91:9
frame #7: 0x000056220827b087 reth-ac3b809190595c73`reth_primitives::transaction::legacy::TxLegacy::encode_fields::h4467f863b49f5f72 [inlined] _$LT$bytes..bytes..Bytes$u20$as$u20$alloy_rlp..encode..Encodable$GT$::encode::he5497625d03f6d9e at encode.rs:220:17
frame #8: 0x000056220827b07d reth-ac3b809190595c73`reth_primitives::transaction::legacy::TxLegacy::encode_fields::h4467f863b49f5f72(self=<unavailable>, out=&mut dyn bytes::buf::buf_mut::BufMut @ 0x00000000383ffb00) at legacy.rs:82:22
frame #9: 0x000056220827eb45 reth-ac3b809190595c73`reth_primitives::transaction::legacy::TxLegacy::encode_with_signature::hf626b7f6627e7213(self=0x00007feb5e19a440, signature=0x00007feb5e19a4f0, out=&mut dyn bytes::buf::buf_mut::BufMut @ 0x000000003840b180) at legacy.rs:97:9
frame #10: 0x0000562208053707 reth-ac3b809190595c73`_$LT$reth_eth_wire_types..message..EthMessage$u20$as$u20$alloy_rlp..encode..Encodable$GT$::encode::hd6a1184fd4cc3499 [inlined] _$LT$reth_primitives..transaction..pooled..PooledTransactionsElement$u20$as$u20$alloy_rlp..encode..Encodable$GT$::encode::h9d9d018da0673fbe(self=0x00007feb5e19a440, out=&mut dyn bytes::buf::buf_mut::BufMut @ 0x000000003877cdd0) at pooled.rs:482:17
frame #11: 0x00005622080536d7 reth-ac3b809190595c73`_$LT$reth_eth_wire_types..message..EthMessage$u20$as$u20$alloy_rlp..encode..Encodable$GT$::encode::hd6a1184fd4cc3499 [inlined] alloy_rlp::encode::encode_list::h7b7eb7f8e0b4ec5e(values=size=0, out=&mut dyn bytes::buf::buf_mut::BufMut @ 0x00000000383ffe20) at encode.rs:334:9
frame #12: 0x000056220805365c reth-ac3b809190595c73`_$LT$reth_eth_wire_types..message..EthMessage$u20$as$u20$alloy_rlp..encode..Encodable$GT$::encode::hd6a1184fd4cc3499 [inlined] _$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$alloy_rlp..encode..Encodable$GT$::encode::h1e509aa6d47fae90 at encode.rs:205:9
frame #13: 0x000056220805365c reth-ac3b809190595c73`_$LT$reth_eth_wire_types..message..EthMessage$u20$as$u20$alloy_rlp..encode..Encodable$GT$::encode::hd6a1184fd4cc3499 [inlined] reth_eth_wire_types::transactions::_::_$LT$impl$u20$alloy_rlp..encode..Encodable$u20$for$u20$reth_eth_wire_types..transactions..PooledTransactions$GT$::encode::hb2fb9bfc4ebbac85(self=<unavailable>, out=&mut dyn bytes::buf::buf_mut::BufMut @ 0x0000000038851390) at transactions.rs:41:5
frame #14: 0x000056220805365c reth-ac3b809190595c73`_$LT$reth_eth_wire_types..message..EthMessage$u20$as$u20$alloy_rlp..encode..Encodable$GT$::encode::hd6a1184fd4cc3499 [inlined] _$LT$reth_eth_wire_types..message..RequestPair$LT$T$GT$$u20$as$u20$alloy_rlp..encode..Encodable$GT$::encode::h43516c5463a7fee0(self=<unavailable>, out=&mut dyn bytes::buf::buf_mut::BufMut @ 0x0000000038830130) at message.rs:458:9
frame #15: 0x0000562208053581 reth-ac3b809190595c73`_$LT$reth_eth_wire_types..message..EthMessage$u20$as$u20$alloy_rlp..encode..Encodable$GT$::encode::hd6a1184fd4cc3499(self=<unavailable>, out=&mut dyn bytes::buf::buf_mut::BufMut @ 0x00000000388300b0) at message.rs:259:55
frame #16: 0x000056220812d2e2 reth-ac3b809190595c73`alloy_rlp::encode::encode::h8258f546e74a0419 [inlined] _$LT$reth_eth_wire_types..message..ProtocolMessage$u20$as$u20$alloy_rlp..encode..Encodable$GT$::encode::hb9b13ca53cf9b0d8(self=0x00007fec0e3fdcc0, out=&mut dyn bytes::buf::buf_mut::BufMut @ 0x000000003886f2a0) at message.rs:124:9
frame #17: 0x000056220812d2c9 reth-ac3b809190595c73`alloy_rlp::encode::encode::h8258f546e74a0419(value=ProtocolMessage @ 0x00007fec0e3fdcc0) at encode.rs:294:5
frame #18: 0x0000562208142db4 reth-ac3b809190595c73`_$LT$reth_network..session..active..ActiveSession$u20$as$u20$core..future..future..Future$GT$::poll::hd93fab6f73483dcc [inlined] _$LT$reth_eth_wire..ethstream..EthStream$LT$S$GT$$u20$as$u20$futures_sink..Sink$LT$reth_eth_wire_types..message..EthMessage$GT$$GT$::start_send::h4d0be6fc373cbd50(self=Pin<&mut reth_eth_wire::ethstream::EthStream<reth_eth_wire::p2pstream::P2PStream<reth_ecies::stream::ECIESStream<tokio::net::tcp::stream::TcpStream>>>> @ scalar, item=EthMessage @ 0x00007fec0e3fe788) at ethstream.rs:321:37
frame #19: 0x0000562208142d34 reth-ac3b809190595c73`_$LT$reth_network..session..active..ActiveSession$u20$as$u20$core..future..future..Future$GT$::poll::hd93fab6f73483dcc [inlined] futures_sink::if_alloc::_$LT$impl$u20$futures_sink..Sink$LT$Item$GT$$u20$for$u20$alloc..boxed..Box$LT$S$GT$$GT$::start_send::h3a5efa773515909c(self=<unavailable>, item=EthMessage @ 0x00007fec0e3fe788) at lib.rs:223:13
frame #20: 0x0000562208142d34 reth-ac3b809190595c73`_$LT$reth_network..session..active..ActiveSession$u20$as$u20$core..future..future..Future$GT$::poll::hd93fab6f73483dcc [inlined] _$LT$reth_network..session..conn..EthRlpxConnection$u20$as$u20$futures_sink..Sink$LT$reth_eth_wire_types..message..EthMessage$GT$$GT$::start_send::h0fb04eaf634a5d92(self=Pin<&mut reth_network::session::conn::EthRlpxConnection> @ 0x00007fec0e3fd838, item=EthMessage @ 0x00007fec0e3fe788) at conn.rs:108:37
frame #21: 0x0000562208142d30 reth-ac3b809190595c73`_$LT$reth_network..session..active..ActiveSession$u20$as$u20$core..future..future..Future$GT$::poll::hd93fab6f73483dcc [inlined] futures_util::sink::SinkExt::start_send_unpin::hcea34e508539f1ac at mod.rs:315:9
frame #22: 0x0000562208142d30 reth-ac3b809190595c73`_$LT$reth_network..session..active..ActiveSession$u20$as$u20$core..future..future..Future$GT$::poll::hd93fab6f73483dcc(self=Pin<&mut reth_network::session::active::ActiveSession> @ 0x00007fec0e3fd838, cx=0x00007fec0e3fe958) at active.rs:557:64
frame #23: 0x0000562208344a0f reth-ac3b809190595c73`_$LT$tracing_futures..Instrumented$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hb9fef2010165e349 [inlined] _$LT$core..pin..Pin$LT$P$GT$$u20$as$u20$core..future..future..Future$GT$::poll::h79d027fa4210745f(self=<unavailable>, cx=0x00007fec0e3fe958) at future.rs:123:9
frame #24: 0x0000562208344a09 reth-ac3b809190595c73`_$LT$tracing_futures..Instrumented$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hb9fef2010165e349 [inlined] _$LT$core..pin..Pin$LT$P$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hfdf3213835eba131(self=<unavailable>, cx=0x00007fec0e3fe958) at future.rs:123:9
frame #25: 0x0000562208344a02 reth-ac3b809190595c73`_$LT$tracing_futures..Instrumented$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hb9fef2010165e349 [inlined] futures_util::future::future::FutureExt::poll_unpin::h2865d935cb7e2759(self=<unavailable>, cx=0x00007fec0e3fe958) at mod.rs:558:9
frame #26: 0x0000562208344a02 reth-ac3b809190595c73`_$LT$tracing_futures..Instrumented$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hb9fef2010165e349 [inlined] _$LT$futures_util..future..select..Select$LT$A$C$B$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hdb60ddc838c52d12(self=Pin<&mut futures_util::future::select::Select<reth_tasks::shutdown::Shutdown, core::pin::Pin<&mut core::pin::Pin<alloc::boxed::Box<(dyn core::future::future::Future<Output=()> + core::marker::Send), alloc::alloc::Global>>>>> @ scalar, cx=0x00007fec0e3fe958) at select.rs:118:35
frame #27: 0x00005622083449e9 reth-ac3b809190595c73`_$LT$tracing_futures..Instrumented$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hb9fef2010165e349 [inlined] reth_tasks::TaskExecutor::spawn_task_as::_$u7b$$u7b$closure$u7d$$u7d$::h80116c8aff2237e3((null)=0x00007fec0e3fe958) at lib.rs:341:50
frame #28: 0x000056220834497b reth-ac3b809190595c73`_$LT$tracing_futures..Instrumented$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::hb9fef2010165e349(self=Pin<&mut tracing_futures::Instrumented<reth_tasks::{impl#5}::spawn_task_as::{async_block_env#0}<core::pin::Pin<alloc::boxed::Box<(dyn core::future::future::Future<Output=()> + core::marker::Send), alloc::alloc::Global>>>>> @ scalar, cx=0x00007fec0e3fe958) at lib.rs:283:9
frame #29: 0x0000562208345167 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h996ea32e7c2a1b83 [inlined] tokio::runtime::task::core::Core$LT$T$C$S$GT$::poll::_$u7b$$u7b$closure$u7d$$u7d$::hc7c3e61697480e04(ptr=0x00007febb2c49030) at core.rs:328:17
frame #30: 0x000056220834515f reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h996ea32e7c2a1b83 [inlined] tokio::loom::std::unsafe_cell::UnsafeCell$LT$T$GT$::with_mut::hf1dd6beefa615389 at unsafe_cell.rs:16:9
frame #31: 0x000056220834515f reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h996ea32e7c2a1b83 [inlined] tokio::runtime::task::core::Core$LT$T$C$S$GT$::poll::h0c4d1cfe5edf34d9(self=0x00007febb2c49020, cx=Context @ 0x00007fec0e3fe958) at core.rs:317:30
frame #32: 0x000056220834515f reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h996ea32e7c2a1b83 [inlined] tokio::runtime::task::harness::poll_future::_$u7b$$u7b$closure$u7d$$u7d$::hb03321bd01310786 at harness.rs:485:19
frame #33: 0x000056220834515f reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h996ea32e7c2a1b83 [inlined] _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h79ba8d93cb17f81b(self=AssertUnwindSafe<tokio::runtime::task::harness::poll_future::{closure_env#0}<tracing_futures::Instrumented<reth_tasks::{impl#5}::spawn_task_as::{async_block_env#0}<core::pin::Pin<alloc::boxed::Box<(dyn core::future::future::Future<Output=()> + core::marker::Send), alloc::alloc::Global>>>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>>> @ 0x0000000038bdab60, (null)=<unavailable>) at unwind_safe.rs:272:9
frame #34: 0x000056220834515f reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h996ea32e7c2a1b83 [inlined] std::panicking::try::do_call::h35103aa86090761b(data=<unavailable>) at panicking.rs:559:40
frame #35: 0x000056220834515f reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h996ea32e7c2a1b83 [inlined] std::panicking::try::hebc18fe529387dcb(f=AssertUnwindSafe<tokio::runtime::task::harness::poll_future::{closure_env#0}<tracing_futures::Instrumented<reth_tasks::{impl#5}::spawn_task_as::{async_block_env#0}<core::pin::Pin<alloc::boxed::Box<(dyn core::future::future::Future<Output=()> + core::marker::Send), alloc::alloc::Global>>>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>>> @ 0x0000000038bd2b90) at panicking.rs:523:19
frame #36: 0x000056220834515f reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h996ea32e7c2a1b83 [inlined] std::panic::catch_unwind::hab2a4f18000fd1e1 at panic.rs:149:14
frame #37: 0x000056220834515f reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h996ea32e7c2a1b83 [inlined] tokio::runtime::task::harness::poll_future::he7a58faf51d1d5f9(core=0x00007febb2c49020, cx=Context @ 0x0000000038be27e0) at harness.rs:473:18
frame #38: 0x000056220834515f reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h996ea32e7c2a1b83 [inlined] tokio::runtime::task::harness::Harness$LT$T$C$S$GT$::poll_inner::h944fdaa1d5efcc25(self=<unavailable>) at harness.rs:208:27
frame #39: 0x0000562208345041 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h996ea32e7c2a1b83 [inlined] tokio::runtime::task::harness::Harness$LT$T$C$S$GT$::poll::h2e9cb744752b1ea4(self=Harness<tracing_futures::Instrumented<reth_tasks::{impl#5}::spawn_task_as::{async_block_env#0}<core::pin::Pin<alloc::boxed::Box<(dyn core::future::future::Future<Output=()> + core::marker::Send), alloc::alloc::Global>>>>, alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle, alloc::alloc::Global>> @ scalar) at harness.rs:153:15
frame #40: 0x0000562208345041 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h996ea32e7c2a1b83(ptr=NonNull<tokio::runtime::task::core::Header> @ scalar) at raw.rs:271:5
frame #41: 0x00005622085e66ee reth-ac3b809190595c73`tokio::runtime::scheduler::multi_thread::worker::Context::run_task::ha53fe4bc40e719d2 [inlined] tokio::runtime::task::raw::RawTask::poll::h204794a7e5dbc5af(self=RawTask @ scalar) at raw.rs:201:18
frame #42: 0x00005622085e66e5 reth-ac3b809190595c73`tokio::runtime::scheduler::multi_thread::worker::Context::run_task::ha53fe4bc40e719d2 [inlined] tokio::runtime::task::LocalNotified$LT$S$GT$::run::h761f925724fe2b86 at mod.rs:427:9
frame #43: 0x00005622085e66e5 reth-ac3b809190595c73`tokio::runtime::scheduler::multi_thread::worker::Context::run_task::ha53fe4bc40e719d2 [inlined] tokio::runtime::scheduler::multi_thread::worker::Context::run_task::_$u7b$$u7b$closure$u7d$$u7d$::h6e56c687f8a5d5ed at worker.rs:657:22
frame #44: 0x00005622085e66b3 reth-ac3b809190595c73`tokio::runtime::scheduler::multi_thread::worker::Context::run_task::ha53fe4bc40e719d2 [inlined] tokio::runtime::coop::with_budget::h2cdc906c51382a2b(budget=Budget @ 0x0000000038881d60, f={closure_env#0} @ 0x0000000038c10840) at coop.rs:107:5
frame #45: 0x00005622085e6655 reth-ac3b809190595c73`tokio::runtime::scheduler::multi_thread::worker::Context::run_task::ha53fe4bc40e719d2 [inlined] tokio::runtime::coop::budget::haff08ba62e6f716f at coop.rs:73:5
frame #46: 0x00005622085e6655 reth-ac3b809190595c73`tokio::runtime::scheduler::multi_thread::worker::Context::run_task::ha53fe4bc40e719d2(self=0x00007fec0e3feb98, task=<unavailable>, core=<unavailable>) at worker.rs:593:9
frame #47: 0x00005622085ee774 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::scheduler::multi_thread::worker::Context::run::hd51ae32d4aea713e(self=<unavailable>, core=0x00007fec1e47d870) at worker.rs:544:24
frame #48: 0x00005622085ee126 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::scheduler::multi_thread::worker::run::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::h9bd6823d173f1bdc at worker.rs:509:21
frame #49: 0x00005622085ee126 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::context::scoped::Scoped$LT$T$GT$::set::h2bbaca5cc8368045(self=<unavailable>, t=0x00007fec0e3feb90, f={closure_env#0} @ 0x0000000038d8e590) at scoped.rs:40:9
frame #50: 0x00005622085ee126 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::context::set_scheduler::_$u7b$$u7b$closure$u7d$$u7d$::h2078cf9f612b0f7d(c=<unavailable>) at context.rs:180:26
frame #51: 0x00005622085ee126 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] std::thread::local::LocalKey$LT$T$GT$::try_with::h7b5a8fd93790dca6(self=<unavailable>, f={closure_env#0}<(), tokio::runtime::scheduler::multi_thread::worker::run::{closure#0}::{closure_env#0}> @ 0x0000000038d92fd0) at local.rs:283:12
frame #52: 0x00005622085ee126 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] std::thread::local::LocalKey$LT$T$GT$::with::h7618fd6cd037f433 at local.rs:260:9
frame #53: 0x00005622085ee126 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::context::set_scheduler::hd7ddf45cb2ddf4ad(v=0x00007fec0e3feb90, f={closure_env#0} @ 0x0000000038d97580) at context.rs:180:17
frame #54: 0x00005622085ee126 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::scheduler::multi_thread::worker::run::_$u7b$$u7b$closure$u7d$$u7d$::h9cd378f49ab972dd((null)=<unavailable>) at worker.rs:504:9
frame #55: 0x00005622085ee126 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::context::runtime::enter_runtime::h7b7c9c2ff88830e7(handle=0x00007fec0e3fec48, allow_block_in_place=true, f={closure_env#0} @ 0x00007fec0e3feca0) at runtime.rs:65:16
frame #56: 0x00005622085ee126 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::scheduler::multi_thread::worker::run::h534414400693b4e3(worker=<unavailable>) at worker.rs:496:5
frame #57: 0x00005622085ed7db reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::scheduler::multi_thread::worker::Launch::launch::_$u7b$$u7b$closure$u7d$$u7d$::h510bbd4037d79fbb at worker.rs:462:45
frame #58: 0x00005622085ed7db reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] _$LT$tokio..runtime..blocking..task..BlockingTask$LT$T$GT$$u20$as$u20$core..future..future..Future$GT$::poll::h726d388b15eed0f3(self=Pin<&mut tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>> @ scalar, _cx=<unavailable>) at task.rs:42:21
frame #59: 0x00005622085ed783 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::task::core::Core$LT$T$C$S$GT$::poll::_$u7b$$u7b$closure$u7d$$u7d$::h45de1fdbec32353a(ptr=0x00007fec1de71a28) at core.rs:328:17
frame #60: 0x00005622085ed783 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::loom::std::unsafe_cell::UnsafeCell$LT$T$GT$::with_mut::h3d7a235f5c139091 at unsafe_cell.rs:16:9
frame #61: 0x00005622085ed783 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::task::core::Core$LT$T$C$S$GT$::poll::hefc5bc8973490de4(self=0x00007fec1de71a20, cx=<unavailable>) at core.rs:317:30
frame #62: 0x00005622085ed783 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::task::harness::poll_future::_$u7b$$u7b$closure$u7d$$u7d$::h8a25f0824cf6d7c7 at harness.rs:485:19
frame #63: 0x00005622085ed783 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h6fc0016b3109f4ee(self=AssertUnwindSafe<tokio::runtime::task::harness::poll_future::{closure_env#0}<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule>> @ 0x0000000038dc2800, (null)=<unavailable>) at unwind_safe.rs:272:9
frame #64: 0x00005622085ed783 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] std::panicking::try::do_call::hdc8c5013fa89a8d8(data=<unavailable>) at panicking.rs:559:40
frame #65: 0x00005622085ed783 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] std::panicking::try::h13995931d2bcf98d(f=AssertUnwindSafe<tokio::runtime::task::harness::poll_future::{closure_env#0}<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule>> @ 0x0000000038bab2c0) at panicking.rs:523:19
frame #66: 0x00005622085ed783 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] std::panic::catch_unwind::h21566643c8c062f7 at panic.rs:149:14
frame #67: 0x00005622085ed783 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::task::harness::poll_future::h18450587c12e0c32(core=0x00007fec1de71a20, cx=Context @ 0x0000000038baaec0) at harness.rs:473:18
frame #68: 0x00005622085ed783 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::task::harness::Harness$LT$T$C$S$GT$::poll_inner::h78af2d89a87bd2ac(self=<unavailable>) at harness.rs:208:27
frame #69: 0x00005622085ed783 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e [inlined] tokio::runtime::task::harness::Harness$LT$T$C$S$GT$::poll::hc932ed76504125a2(self=Harness<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::{impl#0}::launch::{closure_env#0}>, tokio::runtime::blocking::schedule::BlockingSchedule> @ 0x00007fec0e3feb68) at harness.rs:153:15
frame #70: 0x00005622085ed5d6 reth-ac3b809190595c73`tokio::runtime::task::raw::poll::h338c68276c7a9f6e(ptr=NonNull<tokio::runtime::task::core::Header> @ 0x00007fec0e3feb68) at raw.rs:271:5
frame #71: 0x00005622085d8806 reth-ac3b809190595c73`std::sys_common::backtrace::__rust_begin_short_backtrace::h6bb251a963994b37 [inlined] tokio::runtime::task::raw::RawTask::poll::h204794a7e5dbc5af(self=RawTask @ scalar) at raw.rs:201:18
frame #72: 0x00005622085d87fd reth-ac3b809190595c73`std::sys_common::backtrace::__rust_begin_short_backtrace::h6bb251a963994b37 [inlined] tokio::runtime::task::UnownedTask$LT$S$GT$::run::hc671d601bc0da937(self=UnownedTask<tokio::runtime::blocking::schedule::BlockingSchedule> @ scalar) at mod.rs:464:9
frame #73: 0x00005622085d87fd reth-ac3b809190595c73`std::sys_common::backtrace::__rust_begin_short_backtrace::h6bb251a963994b37 [inlined] tokio::runtime::blocking::pool::Task::run::hd772d58076da1e55(self=Task @ 0x0000000038e23d30) at pool.rs:160:9
frame #74: 0x00005622085d87fd reth-ac3b809190595c73`std::sys_common::backtrace::__rust_begin_short_backtrace::h6bb251a963994b37 [inlined] tokio::runtime::blocking::pool::Inner::run::h5fbf111f0eae7dd2(self=0x00007fec1e433d10, worker_thread_id=23) at pool.rs:518:17
frame #75: 0x00005622085d873e reth-ac3b809190595c73`std::sys_common::backtrace::__rust_begin_short_backtrace::h6bb251a963994b37 [inlined] tokio::runtime::blocking::pool::Spawner::spawn_thread::_$u7b$$u7b$closure$u7d$$u7d$::h8ffb425f1a0f312e at pool.rs:476:13
frame #76: 0x00005622085d8691 reth-ac3b809190595c73`std::sys_common::backtrace::__rust_begin_short_backtrace::h6bb251a963994b37(f={closure_env#0} @ 0x00007fec0e3fee20) at backtrace.rs:155:18
frame #77: 0x00005622085d853a reth-ac3b809190595c73`core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::he51a83795cc1e3b4 [inlined] std::thread::Builder::spawn_unchecked_::_$u7b$$u7b$closure$u7d$$u7d$::_$u7b$$u7b$closure$u7d$$u7d$::hdcf0271f63f78494 at mod.rs:542:17
frame #78: 0x00005622085d8535 reth-ac3b809190595c73`core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::he51a83795cc1e3b4 [inlined] _$LT$core..panic..unwind_safe..AssertUnwindSafe$LT$F$GT$$u20$as$u20$core..ops..function..FnOnce$LT$$LP$$RP$$GT$$GT$::call_once::h6d6de407146ce250(self=AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#2}::{closure_env#0}<tokio::runtime::blocking::pool::{impl#6}::spawn_thread::{closure_env#0}, ()>> @ 0x00007fec0e3fee20, (null)=<unavailable>) at unwind_safe.rs:272:9
frame #79: 0x00005622085d8535 reth-ac3b809190595c73`core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::he51a83795cc1e3b4 [inlined] std::panicking::try::do_call::h3778df1caefaaccb(data=<unavailable>) at panicking.rs:559:40
frame #80: 0x00005622085d852a reth-ac3b809190595c73`core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::he51a83795cc1e3b4 [inlined] std::panicking::try::h155f307a8218e1d9(f=<unavailable>) at panicking.rs:523:19
frame #81: 0x00005622085d852a reth-ac3b809190595c73`core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::he51a83795cc1e3b4 [inlined] std::panic::catch_unwind::hb65135f432bc80e7 at panic.rs:149:14
frame #82: 0x00005622085d852a reth-ac3b809190595c73`core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::he51a83795cc1e3b4 [inlined] std::thread::Builder::spawn_unchecked_::_$u7b$$u7b$closure$u7d$$u7d$::h95659dcc1295b20c at mod.rs:541:30
frame #83: 0x00005622085d845e reth-ac3b809190595c73`core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::he51a83795cc1e3b4((null)=0x00007fec1e50b5c0, (null)=<unavailable>) at function.rs:250:5
frame #84: 0x00005622085aa45b reth-ac3b809190595c73`std::sys::pal::unix::thread::Thread::new::thread_start::hb85dbfa54ba503d6 [inlined] _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::hdf5fcef8be77a431 at boxed.rs:2063:9
frame #85: 0x00005622085aa453 reth-ac3b809190595c73`std::sys::pal::unix::thread::Thread::new::thread_start::hb85dbfa54ba503d6 [inlined] _$LT$alloc..boxed..Box$LT$F$C$A$GT$$u20$as$u20$core..ops..function..FnOnce$LT$Args$GT$$GT$::call_once::h8e8c5ceee46ee198 at boxed.rs:2063:9
frame #86: 0x00005622085aa44c reth-ac3b809190595c73`std::sys::pal::unix::thread::Thread::new::thread_start::hb85dbfa54ba503d6 at thread.rs:108:17
frame #87: 0x00007fec1ea916d7 libc.so.6`start_thread + 903
frame #88: 0x00007fec1eb1560c libc.so.6`__clone3 + 44
(lldb)By using nightly build the SIGSEGV does not present.
SeeSoftware
commented
3 weeks ago i got the same thing with v1.0.5
crash at __memcpy_avx512_unaligned_erms () 265 VMOVU (%rsi), %VMM(0) reth_primitives::transaction::legacy::TxLegacy::encode_fields () reth_primitives::transaction::legacy::TxLegacy::encode_with_signature () ...
using RUSTFLAGS="-C target-cpu=native" on AMD cpu rustc 1.80.1 (3f5fd8dd4 2024-08-06)
Rjected
commented
2 weeks ago AMD cpu
@SeeSoftware which AMD cpu specifically?
SeeSoftware
commented
2 weeks ago AMD cpu
@SeeSoftware which AMD cpu specifically?
@Rjected AMD EPYC 9554
jonathanykh
commented
1 week ago same problem on my mini node with amd ryzen 5 4650g. reth v1.0.5, built from source did not use RUSTFLAGS="-C target-cpu=native".
I never had this problem at 1.0.5 before until today I set the priority to Very high on ubuntu's task manager. Will run it with normal priority again to see if it run into such problem again.
update: still encountered segfault after resetting to normal priority and paused livesync by making consensus engine offline. While the node is running i was spamming rpc with cryo.
update2: i think for me, the segfault happens if i do not limit the concurrent requests in cryo, so it seems like spamming rpc could cause this happen. also the rpc i am calling is the trace replay block method. it wasn't like that before in beta/ alpha?
error: reth interrupted by SIGSEGV, printing backtrace
reth(+0x23a1486)[0x561905ee9486]
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f48e3642520]
reth(+0x1887705)[0x5619053cf705]
reth(+0x1081491)[0x561904bc9491]
reth(+0xde7f0a)[0x56190492ff0a]
reth(+0xf5861c)[0x561904aa061c]
reth(+0xdfaa36)[0x561904942a36]
reth(+0xeacded)[0x5619049f4ded]
reth(+0x1ebe245)[0x561905a06245]
reth(+0x1dd5e78)[0x56190591de78]
reth(+0x18e728e)[0x56190542f28e]
reth(+0x43a790)[0x561903f82790]
reth(+0x96ec1b)[0x5619044b6c1b]
reth(+0x97480a)[0x5619044bc80a]
reth(+0x97239f)[0x5619044ba39f]
reth(+0x2ac8a8b)[0x561906610a8b]
/lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7f48e3694ac3]
/lib/x86_64-linux-gnu/libc.so.6(+0x126850)[0x7f48e3726850]
note: we would appreciate a report at https://github.com/paradigmxyz/reth
Segmentation fault (core dumped)
scheminpete
commented
2 days ago Ran into the same issue (and posted my own issue before finding this thread like an idiot).
Also running with an AMD CPU (Threadripper 7960x), same behavior (mainnet, archive, Ubuntu 24.04.1, not running any options besides metrics). Not sure if the AMD CPU is the cause, but the issue started when I updated from 1.0.1 to 1.0.6.
I noticed that the 1.0.2 update includes this merge: chore: move sigsegv handler to reth-cli-util crate #9421. Could be something there.
Describe the bug
I recently synced an eth mainnet reth node from genesis. On completion of the sync, the node can run for a couple minutes before segfaulting.
This is on an AMD EPYC 4344P running Ubuntu 24.04
Steps to reproduce
Sync a node with these options and let it run:
RUST_LOG=info /home/node/reth/target/maxperf/reth node \ --datadir=/home/node/.reth/data \ --chain=mainnet \ --port=30303 \ --authrpc.jwtsecret /home/node/.secrets/jwt.hex \ --authrpc.addr 127.0.0.1 \ --authrpc.port 8551 \ --rpc-max-connections 500 \ --http \ --http.api "debug,eth,net,trace,txpool,web3,rpc" \ --ws \ --ws.api "debug,eth,net,trace,txpool,web3,rpc" \ --metrics=5005 \ --log.file.directory=/home/node/.reth/logs \ --max-outbound-peers 400 \ --max-inbound-peers 200
Node logs
Platform(s)
Linux (x86)
What version/commit are you on?
reth Version: 1.0.3 Commit SHA: 390f30aadebcdd509e72cc04327c3b854de076a6 Build Timestamp: 2024-07-25T07:56:29.407408899Z Build Features: jemalloc Build Profile: maxperf
What database version are you on?
Current database version: 2 Local database version: 2
Which chain / network are you on?
Eth mainnet
What type of node are you running?
Archive (default)
What prune config do you use, if any?
No response
If you've built Reth from source, provide the full command you used
RUSTFLAGS="-C target-cpu=native" cargo build --profile maxperf
Code of Conduct