Run alongside a substrate-backed chain to index all Blocks, State, and Extrinsic data into PostgreSQL.
The schema for the PostgreSQL database is described in the PDF File at the root of this directory.
Examples for how to use substrate-archive are in the examples/
directory
Extended requirements list found in the wiki
--pruning=archive
The CLI is an easier way to get started with substrate-archive. It provides a batteries-included binary, so that you don't have to write any rust code. All thats required is setting up a PostgreSQL DB, and modifying a config file. More information in the wiki
The node-template CLI (in /bin/node-template-archive) is provided as an example of implementing substrate-archive for your chain.
git clone https://github.com/paritytech/substrate-archive.git
# Set up the databases
source ./substrate-archive/scripts/up.sh # Run ./scripts/down.sh to drop the database
cd substrate-archive/bin/polkadot-archive/
# Start the normal polkadot node with `pruning` set to `archive`
polkadot --chain=polkadot --pruning=archive
# Start up the substrate-archive node. `chain` can be one of `polkadot`, `kusama`, or `westend`.
cargo run --release -- -c test_conf.toml --chain=polkadot
You can access the help dialog via cargo run --release -- --help
. Note that up
and down
scripts are meant for convenience and are not meant to be complete. Look in the wiki for more information about the database setup.
Contributors are welcome!
Read the Doc
You can build the documentation for this crate by running cargo doc
.
More Docs here
too many open files
error.Because of the way a RocksDB Secondary Instance works, it requires that all the files of the primary instance remain open in the secondary instance. This could trigger the error on linux, but simply requires that you raise the max open files limit (ulimit
):
$ docker run --ulimit nofile=90000:90000 <image-tag>
ulimit -a 90000
Permanantly Per-User
/etc/security/limits.conf
# /etc/security/limits.conf
* hard nofile 4096
* soft nofile 1024
some_usr hard nofile 90000
some_usr soft nofile 90000
insipx hard nofile 90000
insipx soft nofile 90000
root hard nofile 90000
root soft nofile 90000
For macOS and Linux, a warning message will be raised on the startup when there is a low fd sources limit in the current system, but Windows won't have such a low fd limit warning.
You can contact us at: