search

node-real / bsc-erigon

Ethereum implementation on the efficiency frontier
GNU Lesser General Public License v3.0
124 stars 42 forks source link

How to solve this error "MDBX_MAP_FULL: Environment mapsize limit reached" #201

Closed WangXiaopai595 closed 1 year ago

WangXiaopai595 commented 1 year ago 
[INFO] [08-19|17:08:17.328] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=0000000000000000000000003468ab40d6c4fd3d2bc4dd23a55da23f0e6681a8ffffffff
[INFO] [08-19|17:08:45.326] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=00000000000000000000000035a997d0e3865d8cfc87d00e0e8696f9a045de0affffffff
[INFO] [08-19|17:09:15.324] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=0000000000000000000000003777d08013715da44d97b7ff6c5b81071962e0c1ffffffff
[INFO] [08-19|17:09:45.324] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=00000000000000000000000038d7972e087200252b7b811da35c9e4d2b8566adffffffff
[INFO] [08-19|17:10:15.325] [13/15 LogIndex] ETL [2/2] Loading       into=LogTopicIndex current_prefix=0000000000000000000000003abe5ecbf9da9aa9c343ff200e6343024e49ea07ffffffff
[INFO] [08-19|17:10:16.626] [p2p] GoodPeers                          eth66=99
[EROR] [08-19|17:10:41.313] Staged Sync                              err="[13/15 LogIndex] loadIntoTable LogTopicIndex: 13/15 LogIndex: put: k=0000000000000000000000003bccd8840bb93d85ceabcc7c0bc70af47d3120f1ffffffff, table: LogTopicIndex, err: mdbx_cursor_put: MDBX_MAP_FULL: Environment mapsize limit reached"
WangXiaopai595 commented 1 year ago

I encountered this error during node synchronization, causing the node to restart synchronization. May I ask how to solve this problem? It has already taken me too much time.

WangXiaopai595 commented 1 year ago

I use rpc to read a large number of eths getLogs and trace block, I'm not sure if it's related to this.

WangXiaopai595 commented 1 year ago

nohup erigon --sentry.drop-useless-peers --p2p.protocol=66 --txpool.disable --db.size.limit 9216000000000 --metrics.addr=0.0.0.0 --log.console.verbosity=info --http --http.addr 0.0.0.0 --http.port 8786 --http.api "eth,net,web3,trace,erigon,engine" --rpc.batch.limit 10000 --db.pagesize=16k --datadir ./data --private.api.addr=localhost:9090 --chain=bsc --metrics --log.dir.path ./data/log >> bsc.log 2>&1 &

Now I have changed the startup parameters and added '-- db. size. limit 921600000000'

blxdyx commented 1 year ago

The default db.size is 10TB, it's work for me. Try to restart the rpcdaemon or increase the db.size

WangXiaopai595 commented 1 year ago

[INFO] [08-22|02:55:47.119] [p2p] GoodPeers eth66=100 [INFO] [08-22|02:56:14.304] [14/15 TxLookup] ETL [2/2] Loading into=BlockTransactionLookup current_prefix=a1adde01 [INFO] [08-22|02:56:44.304] [14/15 TxLookup] ETL [2/2] Loading into=BlockTransactionLookup current_prefix=c21f033f [INFO] [08-22|02:57:14.377] [14/15 TxLookup] ETL [2/2] Loading into=BlockTransactionLookup current_prefix=dfa9136c [INFO] [08-22|02:57:44.304] [14/15 TxLookup] ETL [2/2] Loading into=BlockTransactionLookup current_prefix=f90860d1 [INFO] [08-22|02:57:47.120] [p2p] GoodPeers eth66=100 [INFO] [08-22|02:58:59.706] [14/15 TxLookup] DONE in=39m10.859574414s [INFO] [08-22|02:58:59.864] RPC Daemon notified of new headers from=30988389 to=30989413 hash=0x0c317d770192257f430d96b947a9a7b9b25624b8254a78f3d585d873e14af449 header sending=66.922755ms log sending=119.097µs [INFO] [08-22|02:58:59.865] Timings (slower than 50ms) Headers=1h48m49.589s CumulativeIndex=6.158s BlockHashes=2m24.425s Bodies=56m41.47s Senders=4m2.903s Execution=29h46m16.015s HashState=6h1m31.965s IntermediateHashes=5h11m55.228s CallTraces=2h57m26.838s AccountHistoryIndex=1h47m31.067s StorageHistoryIndex=4h20m22.017s LogIndex=2h32m53.27s TxLookup=39m10.859s [INFO] [08-22|02:58:59.887] [2/15 Headers] Waiting for headers... from=30989413 [WARN] [08-22|02:58:59.908] [snapshots] retire blocks err="DumpBlocks: DumpHeaders: header missed in db: block_num=29500000, hash=d0c6fe75dc2c4363b1ede8bdd07d46650f997b3bb15212b372067315fa73d7b5" fromBlock=29500000 toBlock=30000000 [INFO] [08-22|02:59:19.891] [2/15 Headers] Inserting headers progress=30993658 queue=1 I have encountered a new problem that has caused synchronization to fail again. How can I handle this situation?

blxdyx commented 1 year ago

Use this integration stage_headers --reset

WangXiaopai595 commented 1 year ago 
[INFO] [08-22|23:30:14.088] [2/15 Headers] Wrote block headers       number=31078780 blk/second=0.350 alloc=4.8GB sys=12.0GB
[WARN] [08-22|23:30:14.088] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:14.179] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:14.191] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:14.203] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:14.245] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:15.246] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:15.281] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:15.291] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:15.293] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:15.306] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:15.308] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:15.316] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:15.316] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:15.321] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216
[WARN] [08-22|23:30:15.445] [downloader] Added future link           hash=0x3f6deaf672b4a11dcb7c3b2675156a3060825f61174476a82fdf08fd7472e3b6 height=31078781 timestamp=1692718216

Thank you very much. The synchronization has finally been completed. The node has been experiencing this log continuously. Is this normal?

WangXiaopai595 commented 1 year ago

integration stage_exec --datadir ./data/ --chain bsc --unwind 100 I tried executing this command, but it doesn't seem to solve the problem

WangXiaopai595 commented 1 year ago

I don't have high real-time requirements for node synchronization, I can accept it even if it falls behind hundreds of blocks.

WangXiaopai595 commented 1 year ago

I tried to execute this command and received an error

integration stage_headers --unwind=100 --datadir=./data --chain=bsc

INFO[08-23|09:29:39.079] logging to file system                   log dir=data/logs file prefix=erigon log level=info json=false
INFO[08-23|09:29:43.504] [snapshots] Blocks Stat                  blocks=29500k indices=29500k alloc=2.3GB sys=2.4GB
WARN[08-23|09:29:44.295] [8/15 HashState] promotion backwards from 31083252 to 31082152 
INFO[08-23|09:29:44.296] UnwindTo                                 block=31082052 bad_block_hash=0x0000000000000000000000000000000000000000000000000000000000000000
INFO[08-23|09:29:50.189] [10/15 CallTraces] Unwind                from=31082152 to=31082052
INFO[08-23|09:29:51.490] [8/15 HashState] Unwinding started       from=31083252 to=31082052 storage=false codes=true
INFO[08-23|09:29:51.633] [8/15 HashState] Unwinding started       from=31083252 to=31082052 storage=false codes=false
INFO[08-23|09:29:51.956] [8/15 HashState] Unwinding started       from=31083252 to=31082052 storage=true codes=false
INFO[08-23|09:29:53.199] [9/15 IntermediateHashes] Unwinding      from=31083252 to=31082052 csbucket=AccountChangeSet
INFO[08-23|09:29:53.486] [9/15 IntermediateHashes] Unwinding      from=31083252 to=31082052 csbucket=StorageChangeSet
EROR[08-23|09:29:59.429] [9/15 IntermediateHashes] wrong trie root: a89a37a7ab95a67e404516c100c5bc9b4679ec244ce0b62907122aa313f862c7, expected (from header): 297a3883e22c75531f8a8888590530457f9f9d484dc6c2c7db7ae8b14cd61d21
blxdyx commented 1 year ago

show integration print_stages

WangXiaopai595 commented 1 year ago 
INFO[08-23|10:02:54.994] logging to file system                   log dir=data/logs file prefix=erigon log level=info json=false
INFO[08-23|10:02:59.625] [snapshots] Blocks Stat                  blocks=29500k indices=29500k alloc=2.5GB sys=2.7GB
Note: prune_at doesn't mean 'all data before were deleted' - it just mean stage.Prune function were run to this block. Because 1 stage may prune multiple data types to different prune distance.

                 stage_at    prune_at
Snapshots            31083152    0
Headers              31083152    0
BlockHashes              31083252    0
Bodies               31083152    0
Senders              31083252    0
Execution            31082152    31083252
Translation              0       0
HashState            31083252    0
IntermediateHashes       31083252    31083252
AccountHistoryIndex          31083252    0
StorageHistoryIndex          31083252    0
LogIndex             31083252    0
CallTraces           31082152    31083252
TxLookup             31083252    30359000
Finish               31083252    0
--
prune distance: 

blocks.v2: blocks=29499999, segments=29499999, indices=29499999

history.v3: false, idx steps: 0.00, lastMaxTxNum=0->0, lastBlockInSnap=0

sequence: EthTx=4707307635, NonCanonicalTx=0

in db: first header 29500000, last header 31083152, first body 29500000, last body 31083152
blxdyx commented 1 year ago

so it's stuck at 31083252? please show more log

WangXiaopai595 commented 1 year ago 
[INFO] [08-23|07:39:03.541] Opening Database                         label=chaindata path=/home/erigon/data/chaindata
[INFO] [08-23|07:39:03.541] [db] params: growStep=2GB, mapsSize=9765625MB, shrinkThreshold=-1, pageSize=16KB, label=chaindata, WriteMap=false, Durable=false, NoReadahead=true,  
[INFO] [08-23|07:39:03.547] Initialised chain configuration          config="{ChainID: 56 Ramanujan: 0, Niels: 0, MirrorSync: 5184000, Bruno: 13082000, Euler: 18907621, Gibbs: 23846001, Nano: 21962149, Moran: 22107423, Gibbs: 23846001, Planck: 27281024, Luban: 29020050, Plato: 30720096, Hertz: 31302048 Engine: parlia}" genesis=0x0d21840abff46b96c84b2ac9e10e4f5cdaeb5693cb665db62a2f3b02d2d57b5b
[INFO] [08-23|07:39:03.547] Effective                                prune_flags= snapshot_flags="--snapshots=true" history.v3=false
[INFO] [08-23|07:39:14.419] Initialising Ethereum protocol           network=56
[INFO] [08-23|07:39:15.133] new subscription to logs established 
[INFO] [08-23|07:39:15.132] Starting private RPC server              on=localhost:9090
[INFO] [08-23|07:39:15.137] rpc filters: subscribing to Erigon events 
[INFO] [08-23|07:39:15.142] new subscription to newHeaders established 
[INFO] [08-23|07:39:15.143] Reading JWT secret                       path=/home/erigon/data/jwt.hex
[INFO] [08-23|07:39:15.145] HTTP endpoint opened for Engine API      url=127.0.0.1:8551 ws=true ws.compression=true
[INFO] [08-23|07:39:15.145] HTTP endpoint opened                     url=[::]:8786 ws=false ws.compression=true grpc=false
[INFO] [08-23|07:39:15.152] Started P2P networking                   version=66 self=enode://f07d951b5833cdbe11cd5414b683a9b72730e0d40fb6d5e970496eac5685b578a32867529353b33a1213f798bcddebd9154b22612793e43b73ab2c4265f65f57@127.0.0.1:0 name=erigon/v1.1.6/linux-amd64/go1.20.2
[INFO] [08-23|07:39:15.161] [1/15 Snapshots] Fetching torrent files metadata 
[INFO] [08-23|07:39:15.171] [snapshots] Blocks Stat                  blocks=29500k indices=29500k alloc=3.1GB sys=3.3GB
[INFO] [08-23|07:39:15.172] [2/15 Headers] Waiting for headers...    from=31083252
[INFO] [08-23|07:39:35.175] [2/15 Headers] Inserting headers         progress=31085706 queue=1
[WARN] [08-23|07:39:42.804] [downloader] Added future link           hash=0xc2ec8ba1e5ac0c5b75800fd84051b5aaf4010ab8ab29927a4b7e0ccb0d26a7b8 height=31088463 timestamp=1692747583
[WARN] [08-23|07:39:42.804] [downloader] Added future link           hash=0xc2ec8ba1e5ac0c5b75800fd84051b5aaf4010ab8ab29927a4b7e0ccb0d26a7b8 height=31088463 timestamp=1692747583
[WARN] [08-23|07:39:42.980] [downloader] Added future link           hash=0xc2ec8ba1e5ac0c5b75800fd84051b5aaf4010ab8ab29927a4b7e0ccb0d26a7b8 height=31088463 timestamp=1692747583
[WARN] [08-23|07:39:43.770] [downloader] Added future link           hash=0xec6b8ab32055a620d294ce56fbe92d7cf808ae0b1080840adebc5a774754f7bd height=31088464 timestamp=1692747586
[WARN] [08-23|07:39:43.910] [downloader] Added future link           hash=0xec6b8ab32055a620d294ce56fbe92d7cf808ae0b1080840adebc5a774754f7bd height=31088464 timestamp=1692747586
[WARN] [08-23|07:39:44.076] [downloader] Added future link

This has been the case since starting the node, there are no more logs left

blxdyx commented 1 year ago

add this command:--log.console.verbosity=dbug and show more logs

WangXiaopai595 commented 1 year ago 
[INFO] [08-23|10:25:52.107] Starting Erigon on BSC mainnet... 
[INFO] [08-23|10:25:52.109] Maximum peer count                       ETH=100 total=100
[INFO] [08-23|10:25:52.109] starting HTTP APIs                       APIs=eth,net,web3,trace,erigon,engine
[INFO] [08-23|10:25:52.109] torrent verbosity                        level=WRN
[INFO] [08-23|10:25:54.218] Set global gas cap                       cap=50000000
[INFO] [08-23|10:25:54.231] [Downloader] Runnning with               ipv6-enabled=true ipv4-enabled=true download.rate=16mb upload.rate=4mb
[INFO] [08-23|10:25:54.231] Opening Database                         label=chaindata path=/home/erigon/data/chaindata
[INFO] [08-23|10:25:54.231] [db] params: growStep=2GB, mapsSize=9765625MB, shrinkThreshold=-1, pageSize=16KB, label=chaindata, WriteMap=false, Durable=false, NoReadahead=true,  
[INFO] [08-23|10:25:54.242] Initialised chain configuration          config="{ChainID: 56 Ramanujan: 0, Niels: 0, MirrorSync: 5184000, Bruno: 13082000, Euler: 18907621, Gibbs: 23846001, Nano: 21962149, Moran: 22107423, Gibbs: 23846001, Planck: 27281024, Luban: 29020050, Plato: 30720096, Hertz: 31302048 Engine: parlia}" genesis=0x0d21840abff46b96c84b2ac9e10e4f5cdaeb5693cb665db62a2f3b02d2d57b5b
[INFO] [08-23|10:25:54.242] Effective                                prune_flags= snapshot_flags="--snapshots=true" history.v3=false
[INFO] [08-23|10:26:04.171] Initialising Ethereum protocol           network=56
[INFO] [08-23|10:26:04.996] Starting private RPC server              on=localhost:9090
[INFO] [08-23|10:26:04.996] new subscription to logs established 
[INFO] [08-23|10:26:04.997] rpc filters: subscribing to Erigon events 
[DBUG] [08-23|10:26:04.997] Establishing event subscription channel with the RPC daemon ... 
[INFO] [08-23|10:26:04.997] new subscription to newHeaders established 
[INFO] [08-23|10:26:04.999] Reading JWT secret                       path=/home/erigon/data/jwt.hex
[INFO] [08-23|10:26:04.999] HTTP endpoint opened for Engine API      url=127.0.0.1:8551 ws=true ws.compression=true
[INFO] [08-23|10:26:05.000] HTTP endpoint opened                     url=[::]:8786 ws=false ws.compression=true grpc=false
[DBUG] [08-23|10:26:05.003] Couldn't add port mapping                proto=tcp extport=30303 intport=30303 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
[DBUG] [08-23|10:26:05.003] Couldn't add port mapping                proto=udp extport=30303 intport=30303 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
[DBUG] [08-23|10:26:05.013] QuerySeeds read nodes from the node DB   count=30
[INFO] [08-23|10:26:05.016] Started P2P networking                   version=66 self=enode://f07d951b5833cdbe11cd5414b683a9b72730e0d40fb6d5e970496eac5685b578a32867529353b33a1213f798bcddebd9154b22612793e43b73ab2c4265f65f57@127.0.0.1:0 name=erigon/v1.1.6/linux-amd64/go1.20.2
[DBUG] [08-23|10:26:05.016] QuerySeeds read nodes from the node DB   count=30
[INFO] [08-23|10:26:05.019] [1/15 Snapshots] Fetching torrent files metadata 
[INFO] [08-23|10:26:05.030] [snapshots] Blocks Stat                  blocks=29500k indices=29500k alloc=3.1GB sys=3.4GB
[DBUG] [08-23|10:26:05.030] [1/15 Snapshots] DONE                    in=21.482041ms
[INFO] [08-23|10:26:05.030] [2/15 Headers] Waiting for headers...    from=31091764
[DBUG] [08-23|10:26:05.030] [downloader] Request skeleton            anchors=0 highestInDb=31091764
[DBUG] [08-23|10:26:05.655] [sentry] sendMessageById                 msgcode=3 err="set tcp ip:47666: use of closed network connection"
[DBUG] [08-23|10:26:05.739] [sentry] sendMessageById                 msgcode=3 err="set tcp ip:36282: use of closed network connection"
[DBUG] [08-23|10:26:05.753] [downloader] new anchor too far in the past: 30926983, latest header in db: 31091764 
[DBUG] [08-23|10:26:05.753] [downloader] Request skeleton            anchors=0 highestInDb=31091764
[DBUG] [08-23|10:26:05.753] Header request sent                      req="&{Hash:0x0000000000000000000000000000000000000000000000000000000000000000 Number:31091763 Length:192 Skip:1536 Reverse:false Anchor:<nil>}" peer=5772bda0
[DBUG] [08-23|10:26:05.981] [p2p] Penalized peer                     peerId=f16c7f44 name=erigon/v2.36.0-dev-e...
[DBUG] [08-23|10:26:05.981] Requested removal of peer for empty header response peerId=f16c7f44
[DBUG] [08-23|10:26:06.345] [downloader] new anchor too far in the past: 30926983, latest header in db: 31091764 
[DBUG] [08-23|10:26:06.590] [p2p] Penalized peer                     peerId=262f8037 name=Geth/v1.2.10-90ebe34...
[DBUG] [08-23|10:26:06.590] Requested removal of peer for empty header response peerId=262f8037
[DBUG] [08-23|10:26:07.346] [downloader] Request skeleton            anchors=0 highestInDb=31091764
[DBUG] [08-23|10:26:07.346] Header request sent                      req="&{Hash:0x0000000000000000000000000000000000000000000000000000000000000000 Number:31091763 Length:192 Skip:1536 Reverse:false Anchor:<nil>}" peer=5772bda0
[DBUG] [08-23|10:26:07.519] [p2p] Penalized peer                     peerId=40b15210 name=erigon/v2.36.0-dev-e...
[DBUG] [08-23|10:26:07.520] Requested removal of peer for empty header response peerId=40b15210
[DBUG] [08-23|10:26:08.346] [downloader] Request skeleton            anchors=0 highestInDb=31091764
[DBUG] [08-23|10:26:08.346] Header request sent                      req="&{Hash:0x0000000000000000000000000000000000000000000000000000000000000000 Number:31091763 Length:192 Skip:1536 Reverse:false Anchor:<nil>}" peer=5772bda0
WangXiaopai595 commented 1 year ago

After restarting, it seems to be working properly. I'll run it for a while and see what happens

WangXiaopai595 commented 1 year ago

Sorry, this issue has occurred again. Here is the log information. This is' print_stages'

INFO[08-23|12:14:57.035] logging to file system                   log dir=data/logs file prefix=erigon log level=info json=false
INFO[08-23|12:15:02.441] [snapshots] Blocks Stat                  blocks=29500k indices=29500k alloc=2.4GB sys=2.6GB
Note: prune_at doesn't mean 'all data before were deleted' - it just mean stage.Prune function were run to this block. Because 1 stage may prune multiple data types to different prune distance.

                 stage_at    prune_at
Snapshots            31092098    0
Headers              31093606    0
BlockHashes              31093606    0
Bodies               31092098    0
Senders              31092098    0
Execution            31092098    31092098
Translation              0       0
HashState            31092098    0
IntermediateHashes       31092098    31092098
AccountHistoryIndex          31092098    0
StorageHistoryIndex          31092098    0
LogIndex             31092098    0
CallTraces           31092098    31092098
TxLookup             31092098    30359000
Finish               31092098    0
--
prune distance: 

blocks.v2: blocks=29499999, segments=29499999, indices=29499999

history.v3: false, idx steps: 0.00, lastMaxTxNum=0->0, lastBlockInSnap=0

sequence: EthTx=4708109655, NonCanonicalTx=0

in db: first header 29500000, last header 31093606, first body 29500000, last body 31092098

This is the erigon log

[DBUG] [08-23|12:07:42.299] [14/15 TxLookup] DONE                    in=7.299230138s
[DBUG] [08-23|12:07:42.299] [15/15 Finish] DONE                      in=51.125µs
[DBUG] [08-23|12:07:44.543] [snapshots] Dumping txs                  block num=29537794 alloc=4.2GB sys=7.5GB
[INFO] [08-23|12:07:50.320] RPC Daemon notified of new headers       from=31092582 to=31093606 hash=0x8839ef688e726945cc2c6ed61cf8a5d7927369240504ad0f6c1a076c1ba17ed3 header sending=1.885064ms log sending=1.512µs
[INFO] [08-23|12:07:50.321] Commit cycle                             in=8.018252521s
[INFO] [08-23|12:07:50.321] Timings (slower than 50ms)               Headers=6.556s BlockHashes=1.221s Bodies=2m40s Senders=217ms Execution=1m10.633s HashState=1m2.379s IntermediateHashes=1m51.807s CallTraces=33.157s AccountHistoryIndex=16.186s StorageHistoryIndex=43.79s LogIndex=24.288s TxLookup=7.299s
[INFO] [08-23|12:07:50.321] Tables                                   PlainState=265.9GB AccountChangeSet=597.5GB StorageChangeSet=1.3TB BlockTransaction=77.1GB TransactionLog=2.5TB FreeList=148.0MB ReclaimableSpace=591.9GB
[DBUG] [08-23|12:08:03.656] [snapshots] Dumping txs                  block num=29539133 alloc=4.7GB sys=7.5GB
[INFO] [08-23|12:08:04.173] [p2p] GoodPeers                          eth66=75
[DBUG] [08-23|12:08:10.631] [sentry] sendMessageById                 msgcode=3 err="set tcp 122.193.64.155:51750: use of closed network connection"
[DBUG] [08-23|12:08:23.524] [snapshots] Dumping txs                  block num=29539152 alloc=5.1GB sys=7.5GB
[DBUG] [08-23|12:08:44.919] [snapshots] Dumping txs                  block num=29539171 alloc=5.3GB sys=7.5GB
[DBUG] [08-23|12:09:03.339] [snapshots] Dumping txs                  block num=29539187 alloc=5.5GB sys=7.5GB
[DBUG] [08-23|12:09:05.213] [1/15 Snapshots] DONE                    in=8.217587ms
[INFO] [08-23|12:09:05.213] [2/15 Headers] Waiting for headers...    from=31093606
[DBUG] [08-23|12:09:05.213] [downloader] Request skeleton            anchors=0 highestInDb=31093606
[DBUG] [08-23|12:09:05.214] Header request sent                      req="&{Hash:0x0000000000000000000000000000000000000000000000000000000000000000 Number:31093605 Length:192 Skip:0 Reverse:false Anchor:<nil>}" peer=169e5e71
[DBUG] [08-23|12:09:05.285] [p2p] Penalized peer                     peerId=75b1f53f name=erigon/v1.1.6-dev-c2...
[DBUG] [08-23|12:09:05.285] Requested removal of peer for empty header response peerId=75b1f53f
[DBUG] [08-23|12:09:05.373] [p2p] Penalized peer                     peerId=9ef52502 name=erigon/v2.36.0-dev-e...
[DBUG] [08-23|12:09:05.373] Requested removal of peer for empty header response peerId=9ef52502
[DBUG] [08-23|12:09:05.378] [p2p] Penalized peer                     peerId=11c8720c name=erigon/v2.36.0-dev-e...
[DBUG] [08-23|12:09:05.378] Requested removal of peer for empty header response peerId=11c8720c
[DBUG] [08-23|12:09:05.381] [p2p] Penalized peer                     peerId=db994273 name=erigon/v2.36.0-dev-e...
[DBUG] [08-23|12:09:05.381] Requested removal of peer for empty header response peerId=db994273
[DBUG] [08-23|12:09:05.391] [p2p] Penalized peer                     peerId=16b10093 name=erigon/v2.36.0-dev-e...
[DBUG] [08-23|12:09:05.391] Requested removal of peer for empty header response peerId=16b10093
[DBUG] [08-23|12:09:05.428] [p2p] Penalized peer                     peerId=3096133d name=erigon/v2.36.0-dev-e...
[DBUG] [08-23|12:09:05.428] Requested removal of peer for empty header response peerId=3096133d
[DBUG] [08-23|12:09:05.428] [p2p] Penalized peer                     peerId=35aa3dcb name=erigon/v2.36.0-dev-e...
[DBUG] [08-23|12:09:05.428] Requested removal of peer for empty header response peerId=35aa3dcb
[DBUG] [08-23|12:09:05.428] [p2p] Penalized peer                     peerId=94f9495c name=erigon/v2.36.0-dev-e...
[DBUG] [08-23|12:09:05.428] Requested removal of peer for empty header response peerId=94f9495c
[DBUG] [08-23|12:09:05.466] [p2p] Penalized peer                     peerId=674df710 name=erigon/v2.36.0-dev-e...
[DBUG] [08-23|12:09:05.466] Requested removal of peer for empty header response peerId=674df710
[DBUG] [08-23|12:09:05.538] [p2p] Penalized peer                     peerId=19dde0b3 name=erigon/v2.40.0-dev-3...
[DBUG] [08-23|12:09:05.538] Requested removal of peer for empty header response peerId=19dde0b3
[DBUG] [08-23|12:09:05.540] [p2p] Penalized peer                     peerId=9a923b24 name=erigon/v2.40.0-dev-3...
[DBUG] [08-23|12:09:05.540] Requested removal of peer for empty header response peerId=9a923b24
[DBUG] [08-23|12:09:05.561] [p2p] Penalized peer                     peerId=de154a9a name=erigon/v2.40.0-dev-3...
[DBUG] [08-23|12:09:05.561] Requested removal of peer for empty header response peerId=de154a9a
[DBUG] [08-23|12:09:05.569] [p2p] Penalized peer                     peerId=9f287365 name=erigon/v2.36.0-dev-e...
[DBUG] [08-23|12:09:05.569] Requested removal of peer for empty header response peerId=9f287365
[WARN] [08-23|12:09:05.815] Served                                   conn=192.168.1.58:55270 method=trace_block reqid=1 t=1m13.967959367s err=stopped
[DBUG] [08-23|12:09:05.828] [p2p] Penalized peer                     peerId=a50001ad name=erigon/v2.40.0-dev-3...
[DBUG] [08-23|12:09:05.828] Requested removal of peer for empty header response peerId=a50001ad
[WARN] [08-23|12:09:05.889] [downloader] Added future link           hash=0x30846fc62c3791134a0528605702404df7ade1cc2cb5654813085478eabbab2c height=31093830 timestamp=1692763747
[WARN] [08-23|12:09:05.889] [downloader] Added future link           hash=0x30846fc62c3791134a0528605702404df7ade1cc2cb5654813085478eabbab2c height=31093830 timestamp=1692763747
[DBUG] [08-23|12:09:05.900] [p2p] Penalized peer                     peerId=b9885d38 name=erigon/v2.40.0-dev-3...
[DBUG] [08-23|12:09:05.900] Requested removal of peer for empty header response peerId=b9885d38
[WARN] [08-23|12:09:05.931] [downloader] Added future link           hash=0x30846fc62c3791134a0528605702404df7ade1cc2cb5654813085478eabbab2c height=31093830 timestamp=1692763747
[WARN] [08-23|12:09:06.080] [downloader] Added future link           hash=0x30846fc62c3791134a0528605702404df7ade1cc2cb5654813085478eabbab2c height=31093830 timestamp=1692763747
[DBUG] [08-23|12:09:06.121] [p2p] Penalized peer                     peerId=3e7e5d51 name=erigon/v2.40.0-dev-3...
[DBUG] [08-23|12:09:06.121] Requested removal of peer for empty header response peerId=3e7e5d51
[WARN] [08-23|12:09:06.183] [downloader] Added future link           hash=0x30846fc62c3791134a0528605702404df7ade1cc2cb5654813085478eabbab2c height=31093830 timestamp=1692763747
[DBUG] [08-23|12:09:06.823] [downloader] Request skeleton            anchors=0 highestInDb=31093829
[DBUG] [08-23|12:09:06.824] Header request sent                      req="&{Hash:0x0000000000000000000000000000000000000000000000000000000000000000 Number:31093828 Length:192 Skip:0 Reverse:false Anchor:<nil>}" peer=bf44c535
[WARN] [08-23|12:09:06.824] [downloader] Added future link           hash=0x30846fc62c3791134a0528605702404df7ade1cc2cb5654813085478eabbab2c height=31093830 timestamp=1692763747
[WARN] [08-23|12:09:06.859] [downloader] Added future link           hash=0x30846fc62c3791134a0528605702404df7ade1cc2cb5654813085478eabbab2c height=31093830 timestamp=1692763747
WangXiaopai595 commented 1 year ago

I switched the erigon version to the develop branch and it returned to normal