search

openethereum / parity-ethereum

The fast, light, and robust client for Ethereum-like networks.
Other
6.82k stars 1.69k forks source link

Parity DB will not sync on external SSD #11750

Open Yonatan19981 opened 4 years ago

Yonatan19981 commented 4 years ago

Before filing a new issue, please provide the following information.

If you think that your issue is an exploitable security vulnerability, please mail your bugreport to security@parity.io instead; your submission might be eligible for our Bug Bounty. You can find mode info on the reporting process in SECURITY.md

Your issue description goes here below. Try to include actual vs. expected behavior and steps to reproduce the issue.

After trying to sync parity db to an external thunderbolt 3 SSD it got stuck with this error. I restarted my mac, deleted the DB and reinstalled it but it is still reoccurring. The expected behaviour is syncing but the results are as follows. The reason I use an external SSD is because I only have 256 GB in my mac , the DB is bigger and I want to use tracing so I cannot use light mode.

I looked online but could not find a solution to this problem. I saw similar problems occurred in 2016 but the solution was it was fixed with updates. For me this is still happening.

Maybe if someone can try installing the db on an external SSD it would reproduce.

2020-06-01 23:28:38 Starting OpenEthereum/v3.0.0-stable-b3d54f026-20200527/x86_64-apple-darwin/rustc1.43.1 2020-06-01 23:28:38 Keys path /Volumes/VTK512GB/pairtyDB/keys/ethereum 2020-06-01 23:28:38 DB path /Volumes/VTK512GB/pairtyDB/chains/ethereum/db/906a34e69aec8c0d 2020-06-01 23:28:38 State DB configuration: fast +Trace 2020-06-01 23:28:38 Operating mode: active 2020-06-01 23:28:39 Configured for Ethereum using Ethash engine 2020-06-01 23:28:39 Updater is deprecated and may be removed in a future release. Please see #11696 for details: https://github.com/openethereum/openethereum/issues/11696 2020-06-01 23:28:39 Public node URL: enode://21115f73474bbfcc4fd2914a857b8bf98f2b24bd53ab25a811588e0d547040c1f0e4580039d281da461171b5391963f56e4ae7bdb73bd27de17d9ed46daf8a6a@10.0.0.12:30303 2020-06-01 23:28:40 Updated conversion rate to Ξ1 = US$238.95 (19928456 wei/gas) 2020-06-01 23:28:41 Block import failed for #715476 (0xa156…e601) Error: Execution(Internal("Internal error: Database missing expected key: 0x003da4c959c4cc985dda64a0b93469065ada673e94ec8d0732a27c0f95f799f7")) 2020-06-01 23:28:41
Bad block detected: Execution error: Transaction execution error (Internal error: Database missing expected key: 0x003da4c959c4cc985dda64a0b93469065ada673e94ec8d0732a27c0f95f799f7). RLP: f90324f90218a0e9c11a9b69565a68cf5b8ab9267c2bdba2608c91c71b473a423bd5ab552050c4a01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d4934794d8ddb0070aee305fc05be67ecab7e2979e940404a06ecdb3ec8696230ecc3bb9ee7c22c915f6b269efb9c654f1a901c4e99e1168c9a041fbb3755330741e23d0bc4951f968c22c331b9065157642edd3c3b4bf0ca752a0e1b31ee4fa87d1aabe1332b03899dc5977ec7bdb50f00d9c44b9558c5dc8b627b90100000000000000000000000000000000000000000000000000000000000000400000000010000000000000000000000000000080000000000000000000000000000000000000000000000000000000008020000020000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000080000000000000000000000000000000000000001000000000000000000000000000000000080000000000000008607dd8796a6cf830aead4832fefd883047aba845675437e98d783010400844765746887676f312e352e31856c696e7578a030a66dbf63a5b215d4f711f0a330bcaf0f2d6266079148dfdd48d483cd6acc2988c22dd312874415aaf90105f87083010295850ba43b740083015f9094ff501b27c581c367b563767b50d32b7ffebdbdda8841b0c3a494822000801ba0a0e55de425d1de28ec3ac1ec8dfecd6ca26c8869045ba312ad29f42cac3e3c42a0040abec6f726018419ed05652404f4d459d7f276bd28b9ff4ff83d4032b6bca3f89180850ba43b7400830428b294c7696b27830dd8aa4823a1cba8440c27c36adec488095b167ede26a6d4a4cc445611444f574e4c4f41440000000000000000000000000000000000000000000000001ba0cc5e32d7eeef9b1b99c719036a761ed8bdc8fe113c2c38bffa5f3cb5fefefd40a0bd7f328a1a2573586de0c717a9dfbe1709d840202b465071f8250b795112627dc0 Header: Header { parent_hash: 0xe9c11a9b69565a68cf5b8ab9267c2bdba2608c91c71b473a423bd5ab552050c4, timestamp: 1450525566, number: 715476, author: 0xd8ddb0070aee305fc05be67ecab7e2979e940404, transactions_root: 0x41fbb3755330741e23d0bc4951f968c22c331b9065157642edd3c3b4bf0ca752, uncles_hash: 0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347, extra_data: [215, 131, 1, 4, 0, 132, 71, 101, 116, 104, 135, 103, 111, 49, 46, 53, 46, 49, 133, 108, 105, 110, 117, 120], state_root: 0x6ecdb3ec8696230ecc3bb9ee7c22c915f6b269efb9c654f1a901c4e99e1168c9, receipts_root: 0xe1b31ee4fa87d1aabe1332b03899dc5977ec7bdb50f00d9c44b9558c5dc8b627, log_bloom: 0x00000000000000000000000000000000000000000000000000000000000040000000001000000000000000000000000000008000000000000000000000000000000000000000000000000000000000802000002000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000008000000000000000000000000000000000000000100000000000000000000000000000000008000000000000000, gas_used: 293562, gas_limit: 3141592, difficulty: 8648043964111, seal: [[160, 48, 166, 109, 191, 99, 165, 178, 21, 212, 247, 17, 240, 163, 48, 188, 175, 15, 45, 98, 102, 7, 145, 72, 223, 221, 72, 212, 131, 205, 106, 204, 41], [136, 194, 45, 211, 18, 135, 68, 21, 170]], hash: Some(0xa15639f25e05386f48fbef69448e46f2d37d57285e7701db3c7f2b5460e2e601) } Uncles: Transactions:[Tx 0] UnverifiedTransaction { unsigned: Transaction { nonce: 66197, gas_price: 50000000000, gas: 90000, action: Call(0xff501b27c581c367b563767b50d32b7ffebdbdda), value: 4733498320000000000, data: [] }, v: 27, r: 72775311764417853185963420223551836199930346942737545205471688397466720156738, s: 1828236560883950471314771790354185897897851158160360667159554156623283272867, hash: 0xcd007c2dfbc1f4597cb611c2628161d951104d36b4b8d3f6204699a105537acd } [Tx 1] UnverifiedTransaction { unsigned: Transaction { nonce: 0, gas_price: 50000000000, gas: 272562, action: Call(0xc7696b27830dd8aa4823a1cba8440c27c36adec4), value: 674157303370786516, data: [204, 68, 86, 17, 68, 79, 87, 78, 76, 79, 65, 68, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] }, v: 27, r: 92438255643944277883781664585057694541088553031863668790116271200415015435584, s: 85711866770007707660884960618492304231938316778969927919115172588211112403581, hash: 0xe4cb06a92b40fe28a8403c85b36b8ae4ff17d6ab49eb0d3e61991304e52dc7ac }

*Edit: after trying one more time, I got the same error,

2020-06-01 23:50:25 Verifier #1 WARN client Block import failed for #348901 (0xccde…5c6c) Error: Execution(Internal("Internal error: Database missing expected key: 0x00748b086271312fd6ac1e0662c60ae7404b6e521afb9b7ad6da4cbb12127f88")) 2020-06-01 23:50:25 Verifier #1 ERROR client

The command line after that looks like this:

0 µs

2020-06-01 23:51:25 IO Worker #2 INFO import Syncing #348900 0x2eec…a5d4 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 0 Qed #372493 2/25 peers 126 MiB chain 129 MiB db 0 bytes queue 64 MiB sync RPC: 0 conn, 0 req/s, 0 µs 2020-06-01 23:51:30 IO Worker #0 INFO import Syncing #348900 0x2eec…a5d4 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 0 Qed #372493 5/25 peers 126 MiB chain 129 MiB db 0 bytes queue 67 MiB sync RPC: 0 conn, 0 req/s, 0 µs 2020-06-01 23:51:35 IO Worker #0 INFO import Syncing #348900 0x2eec…a5d4 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 0 Qed #372493 2/25 peers 126 MiB chain 129 MiB db 0 bytes queue 70 MiB sync RPC: 0 conn, 0 req/s, 0 µs 2020-06-01 23:51:40 IO Worker #2 INFO import Syncing #348900 0x2eec…a5d4 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 0 Qed #372492 1/25 peers 126 MiB chain 129 MiB db 0 bytes queue 52 MiB sync RPC: 0 conn, 0 req/s, 0 µs 2020-06-01 23:51:45 IO Worker #1 INFO import Syncing #348900 0x2eec…a5d4 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 0 Qed #372491 2/25 peers 126 MiB chain 129 MiB db 0 bytes queue 52 MiB sync RPC: 0 conn, 0 req/s, 0 µs 2020-06-01 23:51:50 IO Worker #2 INFO import Syncing #348900 0x2eec…a5d4 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 0 Qed #372491 2/25 peers 126 MiB chain 129 MiB db 0 bytes queue 52 MiB sync RPC: 0 conn, 0 req/s, 0 µs 2020-06-01 23:51:55 IO Worker #2 INFO import Syncing #348900 0x2eec…a5d4 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 0 Qed #372491 3/25 peers 126 MiB chain 129 MiB db 0 bytes queue 52 MiB sync RPC: 0 conn, 0 req/s, 0 µs 2020-06-01 23:52:00 IO Worker #3 INFO import Syncing #348900 0x2eec…a5d4 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 0 Qed #372491 2/25 peers 126 MiB chain 129 MiB db 0 bytes queue 52 MiB sync RPC: 0 conn, 0 req/s, 0 µs 2020-06-01 23:52:05 IO Worker #3 INFO import Syncing #348900 0x2eec…a5d4 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 0 Qed #372491 2/25 peers 126 MiB chain 129 MiB db 0 bytes queue 52 MiB sync RPC: 0 conn, 0 req/s, 0 µs 2020-06-01 23:52:10 IO Worker #1 INFO import Syncing #348900 0x2eec…a5d4 0.00 blk/s 0.0 tx/s 0.0 Mgas/s 0+ 0 Qed #372491 3/25 peers 126 MiB chain 129 MiB db 0 bytes queue 53 MiB sync RPC: 0 conn, 0 req/s, 0 µs

I tried leaving it on all night last time it happened but it is still stuck on 0 blk/s

Thanks

adria0 commented 4 years ago

@YONATAN10111, this seems a database corruption, that could happen when IO is abruptly detached or when killing the OE process. Openethereum code does not contain any special logic for external volumes.

Could you try with 2.7.2, please? Parity is super-intensive with disk IO, if the problem persists maybe makes sense to run a low-level check to the SSD.

Yonatan19981 commented 4 years ago

@adria0 First of all thank you for replying. I am now trying to install only the db in the external volume and the keys and other data on the internal SSD to see if it works. If it will not work I will try what you offered and will let you know the results.

Yonatan19981 commented 4 years ago

@adria0 do you have any data on the needed read/write speed for the sync process? It can be a good indicator whether you external SSD can sync or not if this is really the problem. If the problem repeats itself, I will do a speed test for both my internal and my external SSD the show the results for other people to compare and see if they can sync it

Yonatan19981 commented 4 years ago

Ok after trying to sync, a little after block 1,000,000 I get this message:

0: backtrace::backtrace::trace 1: backtrace::capture::Backtrace::new 2: panic_hook::gen_panic_msg 3: panic_hook::set_with::{{closure}} 4: std::panicking::rust_panic_with_hook 5: rust_begin_unwind 6: core::panicking::panic_fmt 7: core::option::expect_failed 8: ethcore_blockchain::blockchain::BlockChain::new 9: ethcore::client::client::Client::new 10: ethcore_service::service::ClientService::start 11: openethereum::run::execute_impl 12: openethereum::main_direct 13: openethereum::main 14: std::rt::lang_start::{{closure}} 15: main

Thread 'main' panicked at 'Genesis hash should always exist', ethcore/blockchain/src/blockchain.rs:179

dvdplm commented 4 years ago

FWIW I use a mac and a Samsung X5 external tb3 ssd and it works very well.

Yonatan19981 commented 4 years ago

Tried with 2.6.8 still same problem. Will try with 2.7.2

Yonatan19981 commented 4 years ago

Also checked external SSD health and read/write speed. Health is good and read speed is above 2GB/s and write speed is over 600Mb/s