the stage of execution too slow

[SuperEdison]

This should only be used in very rare cases e.g. if you are not 100% sure if something is a bug or asking a question that leads to improving the documentation. For general questions please use Erigon's discord. blk only 2-4, forever can't catch up lastest block

[SuperEdison]

[calmbeing]

Stuck at 26605878? Can you provide more logs

[calmbeing]

For bsc testnet, suggest specify bootnode in the command, such as: nohup ./build/bin/erigon --datadir ./data --chain=chapel --port=30303 --http.port=8545 --authrpc.port=8551 --torrent.port=42069 --private.api.addr=127.0.0.1:9090 --http --ws --http.api=web3,net,eth,debug,trace,txpool --http.addr=0.0.0.0 --torrent.download.rate=256mb --bootnodes=enode://0637d1e62026e0c8685b1db0ca1c767c78c95c3fab64abc468d1a64b12ca4b530b46b8f80c915aec96f74f7ffc5999e8ad6d1484476f420f0c10e3d42361914b@52.199.214.252:30311","enode://330d768f6de90e7825f0ea6fe59611ce9d50712e73547306846a9304663f9912bf1611037f7f90f21606242ded7fb476c7285cb7cd792836b8c0c5ef0365855c@18.181.52.189:30311","enode://df1e8eb59e42cad3c4551b2a53e31a7e55a2fdde1287babd1e94b0836550b489ba16c40932e4dacb16cba346bd442c432265a299c4aca63ee7bb0f832b9f45eb@52.51.80.128:30311","enode://0bd566a7fd136ecd19414a601bfdc530d5de161e3014033951dd603e72b1a8959eb5b70b06c87a5a75cbf45e4055c387d2a842bd6b1bd8b5041b3a61bab615cf@34.242.33.165:30311","enode://ecd664250ca19b1074dcfbfb48576a487cc18d052064222a363adacd2650f8e08fb3db9de7a7aecb48afa410eaeb3285e92e516ead01fb62598553aed91ee15e@3.209.122.123:30311","enode://665cf77ca26a8421cfe61a52ac312958308d4912e78ce8e0f61d6902e4494d4cc38f9b0dd1b23a427a7a5734e27e5d9729231426b06bb9c73b56a142f83f6b68@52.72.123.113:30311 &

[SuperEdison]

4c 16gb

i use bsc mainnet not chapel

i need 1h 44mins when the execution stage, is too slow, forever catch up the latest block

[calmbeing]

Because of execution time, execution stage is the most time-consuming stage within all sync stages of Erigon, suggest download mdbx snapshot and restart, it'll be faster. Also, in the future, it'll be possible to download state from bit-torrrent instead of execute locally.

[SuperEdison]

Because of execution time, execution stage is the most time-consuming stage within all sync stages of Erigon, suggest download mdbx snapshot and restart, it'll be faster. Also, in the future, it'll be possible to download state from bit-torrrent instead of execute locally.

you mean delete chaindata mdbx.dat keep snapshots file?and restart erigon?

[calmbeing]

Because of execution time, execution stage is the most time-consuming stage within all sync stages of Erigon, suggest download mdbx snapshot and restart, it'll be faster. Also, in the future, it'll be possible to download state from bit-torrrent instead of execute locally.

you mean delete chaindata mdbx.dat keep snapshots file?and restart erigon?

Download the mdbx.dat with full archive data and replace the current chaindata/mdbx.dat, then restart erigon.

[SuperEdison]

Because of execution time, execution stage is the most time-consuming stage within all sync stages of Erigon, suggest download mdbx snapshot and restart, it'll be faster. Also, in the future, it'll be possible to download state from bit-torrrent instead of execute locally.

you mean delete chaindata mdbx.dat keep snapshots file?and restart erigon?

Download the mdbx.dat with full archive data and replace the current chaindata/mdbx.dat, then restart erigon.

I use the mdbx.dat with full archive data, can't still catch up the latest block

[ty-sentio-xyz]

You might be interested in trying this out: https://github.com/sentioxyz/erigon

This is a custom fork from sentio.xyz, the motivation came from the heavy utilization of cloud disks in our production. We are currently using this for eth-mainnet, bsc-mainnet and polygon-mainnet.

Cloud disks provide same throughout but much higher latency comparing to a local NVMe disk. The current design of erigon is not really friendly to cloud disks, as most sync stages are only leveraging sequential I/O (which is very slow: https://cloud.google.com/compute/docs/disks/optimizing-pd-performance#io-queue-depth).

So we decided to introduce parallel execution within selected stages, to fully utilize I/O queue of cloud disks. These are the main improvements:

• HashState Stage: Parallel data extraction. This improves performance when page cache misses.

• Intermediate Hash Stage: Parallel trie hash calculation, by calculating the hash of 16 subtrees then merge. This could improve speed of that stage significantly.

• LogIndex/CallTraceIndex/AccountHistory/StorageHistory: These stages tend to be very slow when updating bitmaps. Bitmaps need to be read from mdbx and rewritten, all sequential. We've changed that stage to parallel extract data into collector, then designed another database (bitmapdb2) to offload bitmaps from mdbx. (This requires --bitmapdb2 flag. Once turned on, you cannot use a separated rpcdaemon, as bitmapdb2 cannot be opened by multiple processes. Also you cannot enable otterscan APIs, as the new database do not yet support these.)

Issues/Criticism/PRs are welcome.

[zhy827827]

@ty-sentio-xyz Does it support BSC hard fork?

[SuperEdison]

You might be interested in trying this out: https://github.com/sentioxyz/erigon

This is a custom fork from sentio.xyz, the motivation came from the heavy utilization of cloud disks in our production. We are currently using this for eth-mainnet, bsc-mainnet and polygon-mainnet.

Cloud disks provide same throughout but much higher latency comparing to a local NVMe disk. The current design of erigon is not really friendly to cloud disks, as most sync stages are only leveraging sequential I/O (which is very slow: https://cloud.google.com/compute/docs/disks/optimizing-pd-performance#io-queue-depth).

So we decided to introduce parallel execution within selected stages, to fully utilize I/O queue of cloud disks. These are the main improvements:

• HashState Stage: Parallel data extraction. This improves performance when page cache misses.
• Intermediate Hash Stage: Parallel trie hash calculation, by calculating the hash of 16 subtrees then merge. This could improve speed of that stage significantly.
• LogIndex/CallTraceIndex/AccountHistory/StorageHistory: These stages tend to be very slow when updating bitmaps. Bitmaps need to be read from mdbx and rewritten, all sequential. We've changed that stage to parallel extract data into collector, then designed another database (bitmapdb2) to offload bitmaps from mdbx. (This requires --bitmapdb2 flag. Once turned on, you cannot use a separated rpcdaemon, as bitmapdb2 cannot be opened by multiple processes. Also you cannot enable otterscan APIs, as the new database do not yet support these.)

Issues/Criticism/PRs are welcome.

polygon mainnet and ethereum mainnet fast than bsc mainnet at execution stage

[qinglin89]

same with #66 , will close this one and track performance issue there.