search

erigontech / erigon

Ethereum implementation on the efficiency frontier https://erigon.gitbook.io
GNU Lesser General Public License v3.0
3.14k stars 1.12k forks source link

How to faster sync on mainnet #8828

Open BananaLF opened 11 months ago

BananaLF commented 11 months ago

We started a new Erigon nodes ~ 3 days ago and it's still sync and sync slowed down in time

Information

erigon version: 2.54.0-aeec5221 hareware: aws cpu: 

CPU(s):                8
Model name:            Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz

ram : 32G disk: 

Filesystem     Type  1K-blocks      Used  Available Use% Mounted on
/dev/nvme0n1p1 ext4 2081739676 982968864 1098754428  48% /

I watch the time of between executeblocks is long，how to speed up it。

Log

[INFO] [11-25|02:54:37.584] [4/12 Execution] Executed blocks         number=11403778 blk/s=22.8 tx/s=3657.8 Mgas/s=277.9 gasState=0.16 batch=241.9MB alloc=3.1GB sys=13.7GB
[INFO] [11-25|02:54:53.639] Committed State                          gas reached=93988694947 gasTarget=549755813888
[INFO] [11-25|02:55:23.770] [] ETL [2/2] Loading                     into=PlainCodeHash current_prefix=6281ab20
[INFO] [11-25|02:55:53.770] [] ETL [2/2] Loading                     into=PlainCodeHash current_prefix=c4a64039
[INFO] [11-25|02:56:11.282] Write to db                              progress=0.1M/3.9M current table=IncarnationMap
[INFO] [11-25|02:56:38.868] [p2p] GoodPeers                          eth68=33 eth67=26 eth66=7
[INFO] [11-25|02:56:39.265] [txpool] stat                            pending=0 baseFee=0 queued=30000 alloc=3.5GB sys=13.7GB
[INFO] [11-25|02:56:47.866] [] ETL [2/2] Loading                     into=PlainState current_prefix=af4bc792
[INFO] [11-25|02:57:19.464] [] ETL [2/2] Loading                     into=PlainState current_prefix=d9d51627
[INFO] [11-25|02:57:49.618] [] ETL [2/2] Loading                     into=PlainState current_prefix=ffbb1d3e
[INFO] [11-25|02:57:49.705] Write to db                              progress=1.8M/3.9M current table=Code
[INFO] [11-25|02:58:19.794] [] ETL [2/2] Loading                     into=Code current_prefix=e65ee186
[INFO] [11-25|02:59:38.868] [p2p] GoodPeers                          eth68=33 eth67=26 eth66=7
[INFO] [11-25|02:59:38.914] [txpool] stat                            pending=0 baseFee=0 queued=30000 alloc=3.4GB sys=13.7GB
[INFO] [11-25|03:00:26.716] [4/12 Execution] Executed blocks         number=11404183 blk/s=1.2 tx/s=193.4 Mgas/s=14.1 gasState=0.00 batch=0B alloc=2.6GB sys=13.7GB
[INFO] [11-25|03:00:37.572] [4/12 Execution] Executed blocks         number=11404521 blk/s=31.1 tx/s=6209.4 Mgas/s=382.1 gasState=0.01 batch=9.6MB alloc=3.9GB sys=13.7GB
[INFO] [11-25|03:00:57.588] [4/12 Execution] Executed blocks         number=11405095 blk/s=28.7 tx/s=5654.5 Mgas/s=351.9 gasState=0.02 batch=28.1MB alloc=3.5GB sys=13.7GB
[INFO] [11-25|03:01:17.581] [4/12 Execution] Executed blocks         number=11405651 blk/s=27.8 tx/s=4940.5 Mgas/s=338.7 gasState=0.03 batch=46.9MB alloc=3.0GB sys=13.7GB
[INFO] [11-25|03:01:37.576] [4/12 Execution] Executed blocks         number=11406219 blk/s=28.4 tx/s=5028.9 Mgas/s=346.9 gasState=0.05 batch=66.6MB alloc=4.9GB sys=13.7GB
[INFO] [11-25|03:01:57.588] [4/12 Execution] Executed blocks         number=11406770 blk/s=27.5 tx/s=4916.2 Mgas/s=335.2 gasState=0.06 batch=85.9MB alloc=4.4GB sys=13.7GB
[INFO] [11-25|03:02:17.583] [4/12 Execution] Executed blocks         number=11407313 blk/s=27.2 tx/s=4323.0 Mgas/s=331.9 gasState=0.07 batch=104.8MB alloc=3.7GB sys=13.7GB
[INFO] [11-25|03:02:37.585] [4/12 Execution] Executed blocks         number=11407805 blk/s=24.6 tx/s=4077.5 Mgas/s=299.7 gasState=0.08 batch=121.4MB alloc=2.8GB sys=13.7GB
[INFO] [11-25|03:02:38.868] [p2p] GoodPeers                          eth68=33 eth67=26 eth66=7
[INFO] [11-25|03:02:38.877] [txpool] stat                            pending=0 baseFee=0 queued=30000 alloc=2.9GB sys=13.7GB
[INFO] [11-25|03:02:57.570] [4/12 Execution] Executed blocks         number=11408348 blk/s=27.2 tx/s=4193.9 Mgas/s=331.3 gasState=0.09 batch=140.0MB alloc=4.6GB sys=13.7GB
[INFO] [11-25|03:03:17.600] [4/12 Execution] Executed blocks         number=11408901 blk/s=27.6 tx/s=4175.3 Mgas/s=332.7 gasState=0.10 batch=158.7MB alloc=3.8GB sys=13.7GB
[INFO] [11-25|03:03:37.591] [4/12 Execution] Executed blocks         number=11409413 blk/s=25.6 tx/s=4089.7 Mgas/s=312.1 gasState=0.12 batch=176.3MB alloc=3.0GB sys=13.7GB
[INFO] [11-25|03:03:57.633] [4/12 Execution] Executed blocks         number=11409988 blk/s=28.7 tx/s=5151.1 Mgas/s=350.5 gasState=0.13 batch=195.6MB alloc=4.9GB sys=13.7GB
[INFO] [11-25|03:04:17.600] [4/12 Execution] Executed blocks         number=11410540 blk/s=27.6 tx/s=4999.7 Mgas/s=341.0 gasState=0.14 batch=215.2MB alloc=4.2GB sys=13.7GB
[INFO] [11-25|03:04:37.577] [4/12 Execution] Executed blocks         number=11411134 blk/s=29.7 tx/s=5755.0 Mgas/s=362.1 gasState=0.15 batch=235.5MB alloc=3.7GB sys=13.7GB
[INFO] [11-25|03:04:57.582] [4/12 Execution] Executed blocks         number=11411690 blk/s=27.8 tx/s=5601.6 Mgas/s=339.4 gasState=0.17 batch=255.0MB alloc=3.0GB sys=13.7GB
[INFO] [11-25|03:04:58.608] Committed State                          gas reached=91955952384 gasTarget=549755813888
[INFO] [11-25|03:05:31.432] [] ETL [2/2] Loading                     into=PlainState current_prefix=e72752f5
[INFO] [11-25|03:05:38.868] [p2p] GoodPeers                          eth68=33 eth66=7 eth67=26
[INFO] [11-25|03:05:38.888] [txpool] stat                            pending=0 baseFee=0 queued=30000 alloc=4.2GB sys=13.7GB
[INFO] [11-25|03:05:45.510] Write to db                              progress=1.6M/4.0M current table=Code
[INFO] [11-25|03:06:15.525] [] ETL [2/2] Loading                     into=Code current_prefix=d5dc7cf5
[INFO] [11-25|03:06:21.464] Write to db                              progress=1.6M/4.0M current table=PlainCodeHash
[INFO] [11-25|03:06:51.490] [] ETL [2/2] Loading                     into=PlainCodeHash current_prefix=0b226cb4
[INFO] [11-25|03:07:21.490] [] ETL [2/2] Loading                     into=PlainCodeHash current_prefix=19feebbf
[INFO] [11-25|03:07:51.480] [] ETL [2/2] Loading                     into=PlainCodeHash current_prefix=5cc8a292
[INFO] [11-25|03:07:58.934] Write to db                              progress=1.6M/4.0M current table=IncarnationMap
[INFO] [11-25|03:08:38.869] [p2p] GoodPeers                          eth68=33 eth67=26 eth66=7
[INFO] [11-25|03:09:11.137] [txpool] stat                            pending=0 baseFee=0 queued=30000 alloc=2.6GB sys=13.7GB
[INFO] [11-25|03:09:11.150] [4/12 Execution] Executed blocks         number=11411717 blk/s=0.1 tx/s=24.2 Mgas/s=1.3 gasState=0.00 batch=0B alloc=2.6GB sys=13.7GB
mans17 commented 11 months ago

Hi, sync can be VERY long. Execution is mùost of the time, though.

https://github.com/ledgerwatch/erigon/blob/devel/eth/stagedsync/README.md

BananaLF commented 11 months ago

Hi, sync can be VERY long. Execution is mùost of the time, though.

https://github.com/ledgerwatch/erigon/blob/devel/eth/stagedsync/README.md

Thanks for you reply.

Have any way speed up sync? like change some config, upgrade hardware,etc...

mans17 commented 11 months ago

First, you need to make sure your 'pruning' is fine since it CANNOT be changed. Then, it's mostly a CPU and I/O on your drive. RAM, big CPU and fast drive will help. Some people fine tune the mempool during sync, it might help a bit.

BananaLF commented 11 months ago

First, you need to make sure your 'pruning' is fine since it CANNOT be changed. Then, it's mostly a CPU and I/O on your drive. RAM, big CPU and fast drive will help. Some people fine tune the mempool during sync, it might help a bit.

Thank you. how to change mempool? If i change --bathSize or --etl.bufferSize --etl.bufferSize, may these config can help speed up sync?

yorickdowne commented 11 months ago

On a Potato (6 cores, 32 GiB RAM, 2TB NVMe) my last Erigon sync took 9 days. For reference.