search

erigontech / erigon

Ethereum implementation on the efficiency frontier https://erigon.gitbook.io
GNU Lesser General Public License v3.0
3.12k stars 1.11k forks source link

If state files are ahead of blocks files - Caplin requesting Erigon to unwind in first FCU (but erigon has no diffs) #10829

Closed AskAlexSharov closed 3 months ago

AskAlexSharov commented 3 months ago

First FCU failing to unwind (because no diffs): 

[WARN] [06-23|04:44:19.798] failed to update fork choice             err="execution Client RPC failed to retrieve ForkChoiceUpdate response, err: cannot unwind to block 20129999, lowest unwindable block is 18446744073709551615"

and then node print logs about 2 block nums (with same hashes) in loop:

[DBUG] [06-21|02:36:44.426] UnwindTo                                 block=20136874 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.426] OnBlock: block is validated              block=0x1938593a103c17dabb61bc6d2f1905ff0f91cfdf2e83decb8e00abe1243672b6
[DBUG] [06-21|02:36:44.428] UnwindTo                                 block=20136875 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.429] OnBlock: block is validated              block=0xf42dd1e74cdfe2b4e8446010e6b5da00117e28d88454ce53f9b8037efa6dd370
[DBUG] [06-21|02:36:44.472] UnwindTo                                 block=20136875 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.472] OnBlock: block is validated              block=0xf42dd1e74cdfe2b4e8446010e6b5da00117e28d88454ce53f9b8037efa6dd370
[DBUG] [06-21|02:36:44.489] UnwindTo                                 block=20136874 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.489] OnBlock: block is validated              block=0x1938593a103c17dabb61bc6d2f1905ff0f91cfdf2e83decb8e00abe1243672b6
[DBUG] [06-21|02:36:44.534] UnwindTo                                 block=20136874 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.534] OnBlock: block is validated              block=0x1938593a103c17dabb61bc6d2f1905ff0f91cfdf2e83decb8e00abe1243672b6
[DBUG] [06-21|02:36:44.539] UnwindTo                                 block=20136875 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.539] OnBlock: block is validated              block=0xf42dd1e74cdfe2b4e8446010e6b5da00117e28d88454ce53f9b8037efa6dd370
[DBUG] [06-21|02:36:44.580] UnwindTo                                 block=20136874 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.580] OnBlock: block is validated              block=0x1938593a103c17dabb61bc6d2f1905ff0f91cfdf2e83decb8e00abe1243672b6
[DBUG] [06-21|02:36:44.583] UnwindTo                                 block=20136875 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.583] OnBlock: block is validated              block=0xf42dd1e74cdfe2b4e8446010e6b5da00117e28d88454ce53f9b8037efa6dd370
[DBUG] [06-21|02:36:44.629] UnwindTo                                 block=20136874 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.630] OnBlock: block is validated              block=0x1938593a103c17dabb61bc6d2f1905ff0f91cfdf2e83decb8e00abe1243672b6
[DBUG] [06-21|02:36:44.637] UnwindTo                                 block=20136875 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.637] OnBlock: block is validated              block=0xf42dd1e74cdfe2b4e8446010e6b5da00117e28d88454ce53f9b8037efa6dd370
[DBUG] [06-21|02:36:44.671] UnwindTo                                 block=20136875 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.671] OnBlock: block is validated              block=0xf42dd1e74cdfe2b4e8446010e6b5da00117e28d88454ce53f9b8037efa6dd370
[DBUG] [06-21|02:36:44.697] UnwindTo                                 block=20136874 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.697] OnBlock: block is validated              block=0x1938593a103c17dabb61bc6d2f1905ff0f91cfdf2e83decb8e00abe1243672b6
[DBUG] [06-21|02:36:44.720] UnwindTo                                 block=20136874 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
[DBUG] [06-21|02:36:44.720] OnBlock: block is validated              block=0x1938593a103c17dabb61bc6d2f1905ff0f91cfdf2e83decb8e00abe1243672b6
[DBUG] [06-21|02:36:44.725] UnwindTo                                 block=20136875 stack="[sync.go:151 ethereum_execution.go:180 ethereum_execution.go:245 kv_temporal.go:109 ethereum_execution.go:244 execution_client.go:63 chain_reader.go:327 execution_client_direct.go:55 on_block.go:100 block_service.go:176 block_service.go:228 map.go:477 block_service.go:221 asm_amd64.s:1695]"
tail -100000f  /erigon-logs/erigon-process.log  | grep -v DBU | grep -v -i p2p

[INFO] [06-21|01:47:13.743] [snapshots:download] Stat                blocks=20108k indices=20108k alloc=9.4GB sys=14.8GB
[INFO] [06-21|01:47:13.768] [snapshots:history] Stat                 blocks=20110k txs=2448m txNum2blockNum="1024=14760K,1536=19814K,1552=19968K,1560=20044K,1564=20082K,1566=20100K,1567=20110K" first_history_idx_in_db=20110960 last_comitment_block=20110960 last_comitment_tx_num=2448437500 alloc=9.4GB sys=14.8GB
[INFO] [06-21|01:47:13.771] [1/12 Snapshots] DONE                    in=3m2.843063923s block=20136884
[INFO] [06-21|01:47:18.306] [Caplin] Forward Sync                    app=caplin stage=ForwardSync progress=9342445 distance-from-chain-tip=16m24s estimated-time-remaining=3m9s
[WARN] [06-21|01:47:21.252] Invalid beacon block                     slot=9342438 blockRoot="[65 196 197 52 219 183 134 161 92 24 93 168 115 140 129 15 69 44 22 84 172 111 106 159 64 59 159 154 82 83 118 45]" reason="block not valid"
[WARN] [06-21|01:47:21.252] bad blocks segment received              err="block not valid"
[INFO] [06-21|01:47:27.233] [snapshots:blocks:retire] Stat           blocks=20109k indices=20109k alloc=15.4GB sys=15.9GB
[INFO] [06-21|01:47:39.477] [snapshots:blocks:retire] Stat           blocks=20110k indices=20110k alloc=14.0GB sys=18.7GB
[INFO] [06-21|01:47:40.337] [4/12 Execution] Unwind Execution        from=20136885 to=20136854
unwindv3: 20136885 -> 20136854
[INFO] [06-21|01:47:40.616] aggregator unwind                        step=1569 txUnwindTo=2452659913 stepsRangeInDB="accounts:2.7, storage:2.7, code:2.7, commitment:0.0, logaddrs: 2.7, logtopics: 2.7, tracesfrom: 2.7, tracesto: 2.7"
unwindv3: 20136885 -> 20136854 done within 7.812231423s
[INFO] [06-21|01:47:59.108] State processing progress                slot=4585896 blk/sec=28.93
[INFO] [06-21|01:48:59.161] State processing progress                slot=4587553 blk/sec=27.62
[INFO] [06-21|01:49:00.283] [snapshots:merge] Stat                   blocks=20110k indices=20110k alloc=16.9GB sys=21.7GB
[INFO] [06-21|01:49:06.389] Blob history download finished successfully
[INFO] [06-21|01:49:10.317] [Antiquary] Antiquating blobs            app=caplin from=9200000 to=9342463
[INFO] [06-21|01:49:59.113] State processing progress                slot=4589049 blk/sec=24.93
[INFO] [06-21|01:50:06.035] [mem] memory stats                       Rss=88.2GB Size=0B Pss=86.8GB SharedClean=2.7GB SharedDirty=0B PrivateClean=64.0GB PrivateDirty=21.5GB Referenced=85.5GB Anonymous=21.5GB Swap=0B alloc=12.1GB sys=22.2GB
[INFO] [06-21|01:50:27.690] [snapshots:blocks:retire] Stat           blocks=20120k indices=20120k alloc=15.3GB sys=22.2GB
[INFO] [06-21|01:50:59.771] State processing progress                slot=4591047 blk/sec=33.30
[INFO] [06-21|01:51:58.277] [snapshots:blocks:retire] Stat           blocks=20130k indices=20130k alloc=12.9GB sys=22.2GB
[INFO] [06-21|01:51:59.102] State processing progress                slot=4593170 blk/sec=35.38
[INFO] [06-21|01:52:10.340] [snapshots:blocks:retire] Stat           blocks=20131k indices=20131k alloc=12.6GB sys=22.2GB
[INFO] [06-21|01:52:22.761] [snapshots:blocks:retire] Stat           blocks=20132k indices=20132k alloc=12.8GB sys=22.2GB
[INFO] [06-21|01:52:35.204] [snapshots:blocks:retire] Stat           blocks=20133k indices=20133k alloc=13.1GB sys=22.2GB
[INFO] [06-21|01:52:48.104] [snapshots:blocks:retire] Stat           blocks=20134k indices=20134k alloc=14.4GB sys=22.2GB
[INFO] [06-21|01:52:59.099] State processing progress                slot=4595268 blk/sec=34.97
[INFO] [06-21|01:53:02.296] [snapshots:blocks:retire] Stat           blocks=20135k indices=20135k alloc=16.2GB sys=22.2GB
[INFO] [06-21|01:53:06.117] [mem] memory stats                       Rss=87.4GB Size=0B Pss=86.0GB SharedClean=2.7GB SharedDirty=0B PrivateClean=65.4GB PrivateDirty=19.3GB Referenced=84.7GB Anonymous=19.3GB Swap=0B alloc=17.5GB sys=22.2GB
[INFO] [06-21|01:53:59.108] State processing progress                slot=4598625 blk/sec=55.95
[INFO] [06-21|01:54:59.090] State processing progress                slot=4601509 blk/sec=48.07
[INFO] [06-21|01:55:59.097] State processing progress                slot=4604422 blk/sec=48.55
[INFO] [06-21|01:56:06.049] [mem] memory stats                       Rss=87.1GB Size=0B Pss=85.8GB SharedClean=2.7GB SharedDirty=0B PrivateClean=65.5GB PrivateDirty=18.9GB Referenced=84.4GB Anonymous=18.9GB Swap=0B alloc=14.2GB sys=22.2GB
[INFO] [06-21|01:56:59.127] State processing progress                slot=4607424 blk/sec=50.03
[INFO] [06-21|01:57:59.456] State processing progress                slot=4610232 blk/sec=46.80
[INFO] [06-21|01:58:59.410] State processing progress                slot=4613033 blk/sec=46.68
[INFO] [06-21|01:59:06.095] [mem] memory stats                       Rss=87.4GB Size=0B Pss=86.1GB SharedClean=2.7GB SharedDirty=0B PrivateClean=65.7GB PrivateDirty=19.1GB Referenced=84.5GB Anonymous=19.1GB Swap=0B alloc=9.6GB sys=22.2GB
[INFO] [06-21|01:59:59.124] State processing progress                slot=4615936 blk/sec=48.38
[INFO] [06-21|02:00:59.094] State processing progress                slot=4618925 blk/sec=49.82
[INFO] [06-21|02:01:59.548] State processing progress                slot=4621968 blk/sec=50.72
[INFO] [06-21|02:02:06.108] [mem] memory stats                       Rss=87.6GB Size=0B Pss=86.3GB SharedClean=2.7GB SharedDirty=0B PrivateClean=65.7GB PrivateDirty=19.3GB Referenced=84.6GB Anonymous=19.3GB Swap=0B alloc=14.5GB sys=22.2GB
[INFO] [06-21|02:02:59.109] State processing progress                slot=4624998 blk/sec=50.50
[INFO] [06-21|02:03:59.096] State processing progress                slot=4628197 blk/sec=53.32
[INFO] [06-21|02:04:59.156] State processing progress                slot=4631104 blk/sec=48.45
[INFO] [06-21|02:05:06.056] [mem] memory stats                       Rss=87.9GB Size=0B Pss=86.6GB SharedClean=2.7GB SharedDirty=0B PrivateClean=65.9GB PrivateDirty=19.4GB Referenced=84.9GB Anonymous=19.4GB Swap=0B alloc=16.7GB sys=22.2GB
[INFO] [06-21|02:05:59.125] State processing progress                slot=4634145 blk/sec=50.68
[INFO] [06-21|02:06:59.094] State processing progress                slot=4636873 blk/sec=45.47
[INFO] [06-21|02:07:59.252] State processing progress                slot=4638706 blk/sec=30.55
[INFO] [06-21|02:08:05.965] [mem] memory stats                       Rss=88.1GB Size=0B Pss=86.7GB SharedClean=2.7GB SharedDirty=0B PrivateClean=66.0GB PrivateDirty=19.5GB Referenced=85.1GB Anonymous=19.5GB Swap=0B alloc=10.2GB sys=22.2GB
[INFO] [06-21|02:08:59.091] State processing progress                slot=4640986 blk/sec=38.00
[INFO] [06-21|02:09:59.366] State processing progress                slot=4643414 blk/sec=40.47
[INFO] [06-21|02:10:59.439] State processing progress                slot=4645781 blk/sec=39.45
[INFO] [06-21|02:11:06.025] [mem] memory stats                       Rss=88.2GB Size=0B Pss=86.8GB SharedClean=2.7GB SharedDirty=0B PrivateClean=66.1GB PrivateDirty=19.5GB Referenced=85.2GB Anonymous=19.5GB Swap=0B alloc=13.2GB sys=22.2GB
[INFO] [06-21|02:11:59.145] State processing progress                slot=4648289 blk/sec=41.80
[INFO] [06-21|02:12:59.596] State processing progress                slot=4650764 blk/sec=41.25
[INFO] [06-21|02:13:59.093] State processing progress                slot=4653643 blk/sec=47.98
[INFO] [06-21|02:14:06.011] [mem] memory stats                       Rss=88.3GB Size=0B Pss=86.9GB SharedClean=2.7GB SharedDirty=0B PrivateClean=66.2GB PrivateDirty=19.5GB Referenced=85.3GB Anonymous=19.5GB Swap=0B alloc=11.1GB sys=22.2GB
[INFO] [06-21|02:14:59.501] State processing progress                slot=4656482 blk/sec=47.32
[INFO] [06-21|02:15:59.175] State processing progress                slot=4659169 blk/sec=44.78
[INFO] [06-21|02:16:59.159] State processing progress                slot=4661824 blk/sec=44.25
[INFO] [06-21|02:17:06.033] [mem] memory stats                       Rss=88.2GB Size=0B Pss=86.9GB SharedClean=2.7GB SharedDirty=0B PrivateClean=66.1GB PrivateDirty=19.5GB Referenced=85.2GB Anonymous=19.5GB Swap=0B alloc=16.5GB sys=22.2GB
[INFO] [06-21|02:17:59.507] State processing progress                slot=4664517 blk/sec=44.88
[INFO] [06-21|02:18:59.471] State processing progress                slot=4667121 blk/sec=43.40
[INFO] [06-21|02:19:59.166] State processing progress                slot=4669953 blk/sec=47.20
[INFO] [06-21|02:20:05.935] [mem] memory stats                       Rss=86.7GB Size=0B Pss=85.4GB SharedClean=2.7GB SharedDirty=0B PrivateClean=64.6GB PrivateDirty=19.5GB Referenced=83.7GB Anonymous=19.5GB Swap=0B alloc=12.1GB sys=22.2GB
[INFO] [06-21|02:20:59.151] State processing progress                slot=4672833 blk/sec=48.00
[INFO] [06-21|02:21:59.175] State processing progress                slot=4675905 blk/sec=51.20
[INFO] [06-21|02:22:59.442] State processing progress                slot=4678701 blk/sec=46.60
[INFO] [06-21|02:23:05.888] [mem] memory stats                       Rss=83.0GB Size=0B Pss=82.4GB SharedClean=1.3GB SharedDirty=0B PrivateClean=62.4GB PrivateDirty=19.3GB Referenced=80.6GB Anonymous=19.3GB Swap=0B alloc=14.1GB sys=22.2GB
[INFO] [06-21|02:23:59.410] State processing progress                slot=4681240 blk/sec=42.32
[INFO] [06-21|02:24:59.092] State processing progress                slot=4683754 blk/sec=41.90
[INFO] [06-21|02:25:59.139] State processing progress                slot=4686528 blk/sec=46.23
[INFO] [06-21|02:26:05.988] [mem] memory stats                       Rss=85.4GB Size=0B Pss=84.0GB SharedClean=2.7GB SharedDirty=0B PrivateClean=63.2GB PrivateDirty=19.5GB Referenced=82.3GB Anonymous=19.5GB Swap=0B alloc=12.9GB sys=22.2GB
[INFO] [06-21|02:26:59.137] State processing progress                slot=4689217 blk/sec=44.82
[INFO] [06-21|02:27:59.757] State processing progress                slot=4692000 blk/sec=46.38
[INFO] [06-21|02:28:59.140] State processing progress                slot=4694912 blk/sec=48.53
[INFO] [06-21|02:29:05.958] [mem] memory stats                       Rss=85.5GB Size=0B Pss=84.1GB SharedClean=2.7GB SharedDirty=0B PrivateClean=63.3GB PrivateDirty=19.5GB Referenced=82.4GB Anonymous=19.5GB Swap=0B alloc=12.0GB sys=22.2GB
[INFO] [06-21|02:29:59.546] State processing progress                slot=4697795 blk/sec=48.05
[INFO] [06-21|02:30:59.088] State processing progress                slot=4700581 blk/sec=46.43
[INFO] [06-21|02:31:59.098] State processing progress                slot=4702903 blk/sec=38.70
[INFO] [06-21|02:32:05.989] [mem] memory stats                       Rss=85.9GB Size=0B Pss=84.5GB SharedClean=2.7GB SharedDirty=0B PrivateClean=63.6GB PrivateDirty=19.7GB Referenced=82.8GB Anonymous=19.7GB Swap=0B alloc=14.4GB sys=22.2GB
[INFO] [06-21|02:32:59.197] State processing progress                slot=4705124 blk/sec=37.02
Giulio2002 commented 3 months ago

So log 1:

Ok can avoid the spam.

Log 2:

Block was not marked as valid so naturally your node is stuck, likely unreleated issue to Caplin

AskAlexSharov commented 3 months ago

It happened after i did rm -rf chaindata rm -rf caplin it did try to unwind 1 block (can’t because no diff) and then start spamming logs

Giulio2002 commented 3 months ago

It happened after i did rm -rf chaindata rm -rf caplin it did try to unwind 1 block (can’t because no diff) and then start spamming logs

then not caplin's fault, and that does not look like spam to me

AskAlexSharov commented 3 months ago

After 24hours of work i see in logs:

[INFO] [06-22|19:29:55.039] State processing progress                slot=4993570 blk/sec=42.83
AskAlexSharov commented 3 months ago

what can I do to make this node work? (i did try to do rm -rf chaindata caplin one more time yesterday).

AskAlexSharov commented 3 months ago

I have random idea: maybe erigon did move to snapshots some non-canonical block (and this is reason why Caplin trying to unwind). Now node has v1-020136-020137-headers.seg file, i will try v1-02013* just to take a look.

AskAlexSharov commented 3 months ago

Removal of recent .seg files didn't help. Node still trying to unwind behind block snapshots and fall into long State processing progress

[WARN] [06-23|04:44:19.798] failed to update fork choice             err="execution Client RPC failed to retrieve ForkChoiceUpdate response, err: cannot unwind to block 20129999, lowest unwindable block is 18446744073709551615"

...
[INFO] [06-23|11:25:03.792] State processing progress                slot=3005881 blk/sec=81.97

node name: snapshotter-bm-e3-ethmainnet-n1

Giulio2002 commented 3 months ago

this is not Caplin though, caplin does NOT causes reorg, Caplin tells erigon the correct the chain and Erigon syncs there, If Erigon believes it needs to reorg then i am not sure how to help or how caplin should help

Giulio2002 commented 3 months ago

Said that - it is not a caplin bug, caplin just sends an execution block hash to erigon which is close to what the latest hash is, if erigon says, hey we need to reorg, caplin cannot control that

AskAlexSharov commented 3 months ago

Said that - it is not a caplin bug, caplin just sends an execution block hash to erigon which is close to what the latest hash is, if erigon says, hey we need to reorg, caplin cannot control that

maybe you are right. for now i have 2 mainnet nodes - where on chain-tip i did rm -rf chaindata caplin and can't sync after it. i didn't investigate deeper yet.

AskAlexSharov commented 3 months ago

Ok. Seems it was because state files was ahead of block files. Next command helped for me (delete last 1 step in state files):

 go run ./cmd/erigon snapshots rm-state-snapshots --step=1564-100000 --datadir=/erigon-data/

what API method using Caplin to understand erigon's progress ?

Giulio2002 commented 3 months ago

why are

Ok. Seems it was because state files was ahead of block files. Next command helped for me (delete last 1 step in state files):

 go run ./cmd/erigon snapshots rm-state-snapshots --step=1564-100000 --datadir=/erigon-data/

what API method using Caplin to understand erigon's progress ?

Caplin does not understand erigon progress, it just calls FCU to set progress

Giulio2002 commented 3 months ago

Btw if we realized there is no bug we should not keep this open