Rare SIGSEGV during mainnet benchmark 🥶

risechain / pevm

Blazingly fast Parallel EVM

MIT License

217 stars 41 forks source link

Rare SIGSEGV during mainnet benchmark 🥶 #287

Closed hai-rise closed 1 month ago

hai-rise commented 1 month ago

Benchmarking Block 15274915(1226 txs, 29928443 gas)/Parallel: Collecting 100 samples in estimated 5.2303 s (1900 iterations)error: bench failed, to rerun pass `--bench mainnet`

Caused by:
  process didn't exit successfully: `.../pevm/target/release/deps/mainnet-2bec42578f0ffffc --noplot --bench` (signal: 11, SIGSEGV: invalid memory reference)

hai-rise commented 1 month ago

69c90e62b341102223b54169759a8338a6843207 seems to have caused it. This is only reproducible on ARM, not on x86.

hai-rise commented 1 month ago

This is miraculously fixed by #286. It could have been an unsafe race condition of defer-drop triggered by referencing data in its mv_memory. We need a better garbage collection mechanism to replace defer-drop (last updated 2 years ago). https://github.com/risechain/pevm/blob/c41223f553ad4f4c0fd00d706c140b3b7584eaf3/src/pevm.rs#L118-L119

hai-rise commented 1 month ago

It's very likely to be a rpmalloc issue on Graviton as others don't crash. Current "fix": https://github.com/risechain/pevm/pull/290.