search

paritytech / substrate-archive

Blockchain Indexing Engine
GNU General Public License v3.0
198 stars 72 forks source link

Problem with building #224

Closed alexqrid closed 3 years ago

alexqrid commented 3 years ago

Hi. I am using ubuntu 20.04 LTS VPS. I have successfully set up the archival node and fully synced with the blockchain. But everytime when I run substrated-archive to index all data to the postgres I face with the problem:

Logs ```sh polkadot:~/substrate-archive/bin/polkadot-archive# cargo run --release -- -c /root/polka_conf.toml --chain=polkadot Compiling rococo-runtime v0.8.27 (https://github.com/paritytech/polkadot?branch=master#b2fea426) Compiling polkadot-runtime v0.8.27 (https://github.com/paritytech/polkadot?branch=master#b2fea426) Compiling westend-runtime v0.8.27 (https://github.com/paritytech/polkadot?branch=master#b2fea426) Compiling kusama-runtime v0.8.27 (https://github.com/paritytech/polkadot?branch=master#b2fea426) Compiling polkadot-service v0.8.3 (https://github.com/paritytech/polkadot?branch=master#b2fea426) Compiling polkadot-archive v0.2.2 (/root/substrate-archive/bin/polkadot-archive) Finished release [optimized] target(s) in 6m 27s Running `target/release/polkadot-archive -c /root/polka_conf.toml --chain=polkadot` [12:51] INFO Open RocksDB at /root/.local/share/polkadot/chains/polkadot/db, state column budget: 115 MiB, others(11) column cache: 1 MiB [12:52] INFO Running archive for chain `polkadot` 🔗, implementation `parity-polkadot`. Latest known runtime version: 27. Latest finalized block 0x5412…bc15 🛡️ [12:52] INFO 0 missing blocks [12:52] INFO Restoring 0 missing storage entries. This could take a few minutes... [12:52] INFO Storage restored [12:52] INFO Took 25.053473691s to load 100001 blocks thread 'blocking-1' panicked at 'No storage found for 0x91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3', /root/substrate-archive/substrate-archive-backend/src/read_only_backend.rs:100:66 stack backtrace: 0: rust_begin_unwind at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:495:5 1: std::panicking::begin_panic_fmt at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:437:5 2: substrate_archive_backend::read_only_backend::ReadOnlyBackend::storage::{{closure}} 3: substrate_archive_backend::read_only_backend::ReadOnlyBackend::storage 4: substrate_archive_backend::runtime_version_cache::RuntimeVersionCache::get 5: substrate_archive_backend::runtime_version_cache::RuntimeVersionCache::find_pivot 6: substrate_archive_backend::runtime_version_cache::RuntimeVersionCache::find_versions_as_blocks 7: as core::future::future::Future>::poll 8: async_task::raw::RawTask::run 9: blocking::Executor::main_loop note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. thread 'smol-1' panicked at 'task has failed', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/async-task-4.0.3/src/task.rs:368:45 stack backtrace: 0: rust_begin_unwind at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:495:5 1: core::panicking::panic_fmt at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/panicking.rs:92:14 2: core::option::expect_failed at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/option.rs:1260:5 3: as core::future::future::Future>::poll 4: as core::future::future::Future>::poll 5: as core::future::future::Future>::poll 6: as core::future::future::Future>::poll 7: as core::future::future::Future>::poll 8: async_task::raw::RawTask::run 9: as core::future::future::Future>::poll 10: as core::future::future::Future>::poll 11: async_io::driver::block_on 12: smol::spawn::spawn::GLOBAL::{{closure}}::{{closure}} note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. ```
Full stack trace ```sh [12:57] INFO Running archive for chain `polkadot` 🔗, implementation `parity-polkadot`. Latest known runtime version: 27. Latest finalized block 0x3a73…8067 🛡️ [12:57] INFO 0 missing blocks [12:57] INFO Restoring 0 missing storage entries. This could take a few minutes... [12:57] INFO Storage restored [12:57] INFO Took 25.478931525s to load 100001 blocks thread 'blocking-1' panicked at 'No storage found for 0x91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3', /root/substrate-archive/substrate-archive-backend/src/read_only_backend.rs:100:66 stack backtrace: 0: 0x5614778743a0 - std::backtrace_rs::backtrace::libunwind::trace::h04d12fdcddff82aa at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/../../backtrace/src/backtrace/libunwind.rs:100:5 1: 0x5614778743a0 - std::backtrace_rs::backtrace::trace_unsynchronized::h1459b974b6fbe5e1 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5 2: 0x5614778743a0 - std::sys_common::backtrace::_print_fmt::h9b8396a669123d95 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:67:5 3: 0x5614778743a0 - ::fmt::he009dcaaa75eed60 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:46:22 4: 0x56147789854c - core::fmt::write::h77b4746b0dea1dd3 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/fmt/mod.rs:1078:17 5: 0x56147786e0e2 - std::io::Write::write_fmt::heb7e50902e98831c at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/io/mod.rs:1518:15 6: 0x561477876e15 - std::sys_common::backtrace::_print::h2d880c9e69a21be9 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:49:5 7: 0x561477876e15 - std::sys_common::backtrace::print::h5f02b1bb49f36879 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:36:9 8: 0x561477876e15 - std::panicking::default_hook::{{closure}}::h658e288a7a809b29 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:208:50 9: 0x561477876ab8 - std::panicking::default_hook::hb52d73f0da9a4bb8 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:227:9 10: 0x5614778775b1 - std::panicking::rust_panic_with_hook::hfe7e1c684e3e6462 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:593:17 11: 0x5614778770f7 - std::panicking::begin_panic_handler::{{closure}}::h42939e004b32765c at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:499:13 12: 0x56147787485c - std::sys_common::backtrace::__rust_end_short_backtrace::h9d2070f7bf9fd56c at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:141:18 13: 0x561477877059 - rust_begin_unwind at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:495:5 14: 0x56147787700b - std::panicking::begin_panic_fmt::h4a7e15187eac098d at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:437:5 15: 0x561476ad2e8e - substrate_archive_backend::read_only_backend::ReadOnlyBackend::storage::{{closure}}::hdcadf5909eeb7ab1 16: 0x561476ad2e0a - substrate_archive_backend::read_only_backend::ReadOnlyBackend::storage::hc74656ebee76ad36 17: 0x5614769f80ea - substrate_archive_backend::runtime_version_cache::RuntimeVersionCache::get::h8998100ce8f60739 18: 0x5614769f6b84 - substrate_archive_backend::runtime_version_cache::RuntimeVersionCache::find_pivot::he5e391dcc661e886 19: 0x5614769f7be6 - substrate_archive_backend::runtime_version_cache::RuntimeVersionCache::find_versions_as_blocks::h22d933b68bb6cb1c 20: 0x56147694df67 - as core::future::future::Future>::poll::hb3ebfd1c472d3960 21: 0x5614768e98d2 - async_task::raw::RawTask::run::h97381b54ccd61b4c 22: 0x5614776889c4 - blocking::Executor::main_loop::h346b494b7dbf156e 23: 0x561477689bf6 - std::sys_common::backtrace::__rust_begin_short_backtrace::hc0aa1d5839f4ff41 24: 0x5614776897d4 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h30b3a2a8907a564c 25: 0x56147787b64a - as core::ops::function::FnOnce>::call_once::h09ff301006f1aeca at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/alloc/src/boxed.rs:1307:9 26: 0x56147787b64a - as core::ops::function::FnOnce>::call_once::he79488c8f00b5f31 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/alloc/src/boxed.rs:1307:9 27: 0x56147787b64a - std::sys::unix::thread::Thread::new::thread_start::h587efff279c68ba7 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys/unix/thread.rs:71:17 28: 0x7fa93a310609 - start_thread 29: 0x7fa93a0e0293 - clone 30: 0x0 - thread 'smol-1' panicked at 'task has failed', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/async-task-4.0.3/src/task.rs:368:45 stack backtrace: 0: 0x5614778743a0 - std::backtrace_rs::backtrace::libunwind::trace::h04d12fdcddff82aa at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/../../backtrace/src/backtrace/libunwind.rs:100:5 1: 0x5614778743a0 - std::backtrace_rs::backtrace::trace_unsynchronized::h1459b974b6fbe5e1 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5 2: 0x5614778743a0 - std::sys_common::backtrace::_print_fmt::h9b8396a669123d95 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:67:5 3: 0x5614778743a0 - ::fmt::he009dcaaa75eed60 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:46:22 4: 0x56147789854c - core::fmt::write::h77b4746b0dea1dd3 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/fmt/mod.rs:1078:17 5: 0x56147786e0e2 - std::io::Write::write_fmt::heb7e50902e98831c at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/io/mod.rs:1518:15 6: 0x561477876e15 - std::sys_common::backtrace::_print::h2d880c9e69a21be9 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:49:5 7: 0x561477876e15 - std::sys_common::backtrace::print::h5f02b1bb49f36879 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:36:9 8: 0x561477876e15 - std::panicking::default_hook::{{closure}}::h658e288a7a809b29 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:208:50 9: 0x561477876ab8 - std::panicking::default_hook::hb52d73f0da9a4bb8 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:227:9 10: 0x5614778775b1 - std::panicking::rust_panic_with_hook::hfe7e1c684e3e6462 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:593:17 11: 0x5614778770f7 - std::panicking::begin_panic_handler::{{closure}}::h42939e004b32765c at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:499:13 12: 0x56147787485c - std::sys_common::backtrace::__rust_end_short_backtrace::h9d2070f7bf9fd56c at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:141:18 13: 0x561477877059 - rust_begin_unwind at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:495:5 14: 0x561477896451 - core::panicking::panic_fmt::ha0bb065d9a260792 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/panicking.rs:92:14 15: 0x561477896083 - core::option::expect_failed::h4c033402fb75f2bf at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/option.rs:1260:5 16: 0x561476aad5c0 - as core::future::future::Future>::poll::he9fc9d7b49113208 17: 0x56147696ae38 - as core::future::future::Future>::poll::h673cd35785651154 18: 0x56147699301e - as core::future::future::Future>::poll::h6bb46280dc3c0ff6 19: 0x561476969322 - as core::future::future::Future>::poll::h628653ba32cd3367 20: 0x56147695ff11 - as core::future::future::Future>::poll::h32c6ea949fbc8e0a 21: 0x5614768e5a7b - async_task::raw::RawTask::run::h210495607ffeeb54 22: 0x561476d18200 - as core::future::future::Future>::poll::h430399fbcc208618 23: 0x561476d18331 - as core::future::future::Future>::poll::hbc53b28557dd6985 24: 0x561476d15cce - async_io::driver::block_on::hf0533aa92eb8f1c1 25: 0x561476d183f5 - smol::spawn::spawn::GLOBAL::{{closure}}::{{closure}}::hc8c8bf13c2d6d154 26: 0x561476d17ef6 - std::sys_common::backtrace::__rust_begin_short_backtrace::hde5836b3a5b8d9b4 27: 0x561476d16c60 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h5fcb1cf0ea330544 28: 0x56147787b64a - as core::ops::function::FnOnce>::call_once::h09ff301006f1aeca at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/alloc/src/boxed.rs:1307:9 29: 0x56147787b64a - as core::ops::function::FnOnce>::call_once::he79488c8f00b5f31 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/alloc/src/boxed.rs:1307:9 30: 0x56147787b64a - std::sys::unix::thread::Thread::new::thread_start::h587efff279c68ba7 at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys/unix/thread.rs:71:17 31: 0x7fa93a310609 - start_thread 32: 0x7fa93a0e0293 - clone 33: 0x0 - ```

I haven't change anything in test_conf.toml file except postgres database credentials.

insipx commented 3 years ago

Thanks for your report!

This likely occurs because our RuntimeVersion cache tries to fetch the storage of the Genesis block (0x91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3 is genesis on Polkadot), and it can't find anything. Should be a quick fix

alexqrid commented 3 years ago

So what should I do if I want to run it, wait until the team release the fix or there is a quick fix?

insipx commented 3 years ago

Actually, I am having trouble reproducing this. I am able to get RuntimeVersionInfo polkadot fine even on the genesis block:

~/projects/parity/substrate-archive/bin/polkadot-archive on master •
[I] ❯ ./target/release/polkadot-archive --chain=polkadot -c my_conf.toml
[17:35] INFO Open RocksDB at /home/insipx/.local/share/polkadot/chains/polkadot/db, state column budget: 115 MiB, others(11) column cache: 1 MiB
[17:35] INFO Running archive for 🔗 `polkadot`, implementation `parity-polkadot`. Latest known runtime version: 27. Latest finalized block 0xcddd…e006 🛡️
[17:35] INFO 0 missing blocks
[17:35] INFO Restoring 0 missing storage entries. This could take a few minutes...
[17:35] INFO Storage restored
[17:35] INFO Took 1.13820962s to load 1001 blocks
[17:35] INFO Getting metadata for hash 91b171bb158e2d3848fa23a9f1c25182fb8e20313b2c1eb49219da7a70ce90c3, version 0
[17:35] INFO Took 891.487515ms to insert 1001 blocks
[17:35] INFO Indexing 9 blocks of storage entries
[17:35] INFO Took 1.165941462s to load 1000 blocks
[17:35] INFO Indexing 9 blocks of storage entries
^C[17:35] INFO waiting for last storage insert...
[17:35] INFO Took 1.17366154s to load 1000 blocks
[17:35] INFO storage inserted

Can you confirm that you are running your polkadot node with --pruning=archive ? If so, what other settings do you have polkadot configured with on launch?

alexqrid commented 3 years ago

Oh, my mistake. I thought I was running in archive mode, but suddenly noticed that forgot this flag. Thank you for the quick response.