search

node-real / bsc-erigon

Ethereum implementation on the efficiency frontier
GNU Lesser General Public License v3.0
118 stars 44 forks source link

1.2.12 hotfix wrong trie root #430

Closed temuera closed 2 weeks ago

temuera commented 2 weeks ago

How to fix broken data. Try fix follow those step, one by one, just need unwind 1 block, use this branch: https://github.com/blxdyx/bsc-erigon/tree/for_test_unwind.

  1. make integration && make erigon
  2. ./build/bin/integration state_stages --unwind=1 --chain=bsc --datadir=./data
  3. ./build/bin/integration stage_headers --unwind=1 --chain=bsc --datadir=./data
  4. add --stage.step=1flag to start command and then restart.

If you node back to tip, just stop and switch to the latest main branch, and restart as before.

blxdyx commented 2 weeks ago

If you are in the wrong version: stop and make integration and state_stages unwind and stage_headers unwind. and the check to the latest version and restart.

WangXiaopai595 commented 2 weeks ago

Encountered the same problem

WangXiaopai595 commented 2 weeks ago 
[INFO] [07-10|15:41:38.215] [8/15 HashState] Unwinding started       from=40343181 to=40343180 storage=false codes=true
[INFO] [07-10|15:41:38.217] [8/15 HashState] Unwinding started       from=40343181 to=40343180 storage=false codes=false
[INFO] [07-10|15:41:38.229] [8/15 HashState] Unwinding started       from=40343181 to=40343180 storage=true codes=false
[INFO] [07-10|15:41:38.270] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343180 csbucket=AccountChangeSet
[INFO] [07-10|15:41:38.273] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343180 csbucket=StorageChangeSet
[EROR] [07-10|15:41:38.410] Staged Sync                              err="[9/15 IntermediateHashes] wrong trie root: 8607b05aca7da11632f52ff54220064e43f6d37af8a289feb74735a1940efea7, expected (from header): 5a1525f7a9cdeac03018695fbf4f863a2b4af60179b0c6b0ccf4760abe2f77b0"
WangXiaopai595 commented 2 weeks ago

The downgraded version is also of no use

blxdyx commented 2 weeks ago

Do you try unwind before you restart with the v1.2.12?

WangXiaopai595 commented 2 weeks ago

The following error occurred after unwind

[WARN] [07-10|16:02:31.346] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:31.518] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:31.751] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:31.811] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:31.825] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:32.002] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:32.079] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:32.095] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:32.104] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:32.115] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:32.129] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:32.598] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:33.228] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:33.248] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:33.270] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:33.290] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:33.647] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:33.657] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
[WARN] [07-10|16:02:33.668] [downloader] Rejected header marked as bad hash=0x2545857dcb180ccbe191e50b27e79f3c07797b70fb3d94b70211253cf6acb0f3 height=40343182
saravadeanil commented 2 weeks ago

@blxdyx Will there be any issue if we upgrade to v1.2.12 before the unwind procedure?

Because I upgraded the node and later performed unwind as the steps were not present earlier on the release page. I faced the below error:-

EROR[07-10|08:12:34.492] [9/15 IntermediateHashes] header not found for block number 40343161 
/ $ integration stage_headers --unwind=20  --chain=bsc --datadir=/erigon/data
INFO[07-10|08:12:16.043] logging to file system                   log dir=/erigon/data/logs file prefix=integration log level=info json=false
INFO[07-10|08:12:16.045] [db] open                                label=chaindata sizeLimit=10TB pageSize=8192
INFO[07-10|08:12:18.903] [snapshots:all] Stat                     blocks=34000k indices=34000k alloc=2.5GB sys=2.6GB
INFO[07-10|08:12:18.906] [snapshots:all] Stat                     blocks=0k indices=0k alloc=2.5GB sys=2.6GB
INFO[07-10|08:12:18.924] Opening Database                         label=parlia path=/erigon/data/parlia
mdbx_setup_dxb:16114 opening after an unclean shutdown, but boot-id(af548bd7f59bb113-d34caf1d0d784219) is MATCH: rollback NOT needed, steady-sync NEEDED
INFO[07-10|08:12:18.926] Opening Database                         label=blob path=/erigon/data/blobs/blob
INFO[07-10|08:12:19.357] Progress                                 headers=40343141

/ $ integration state_stages --unwind=20 --chain=bsc --datadir=/erigon/data

INFO[07-10|08:12:29.969] logging to file system                   log dir=/erigon/data/logs file prefix=integration log level=info json=false
INFO[07-10|08:12:29.975] [db] open                                label=chaindata sizeLimit=10TB pageSize=8192
INFO[07-10|08:12:33.038] [snapshots:all] Stat                     blocks=34000k indices=34000k alloc=2.3GB sys=2.4GB
INFO[07-10|08:12:33.041] [snapshots:all] Stat                     blocks=0k indices=0k alloc=2.3GB sys=2.4GB
INFO[07-10|08:12:33.059] Opening Database                         label=parlia path=/erigon/data/parlia
mdbx_setup_dxb:16114 opening after an unclean shutdown, but boot-id(af548bd7f59bb113-d34caf1d0d784219) is MATCH: rollback NOT needed, steady-sync NEEDED
INFO[07-10|08:12:33.061] Opening Database                         label=blob path=/erigon/data/blobs/blob
INFO[07-10|08:12:34.025] [10/15 CallTraces] Unwind                from=40343181 to=40343161
INFO[07-10|08:12:34.195] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=true
INFO[07-10|08:12:34.217] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=false
INFO[07-10|08:12:34.273] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=true codes=false
EROR[07-10|08:12:34.492] [9/15 IntermediateHashes] header not found for block number 40343161
blxdyx commented 2 weeks ago

One way is download snapshots https://github.com/bnb-chain/bsc-snapshots?tab=readme-ov-file#source-3-erigon-archive-node8tb.

zhy827827 commented 2 weeks ago

integration stage_headers --unwind=20 --chain=bsc --datadir=/opt/bscdata/erigon INFO[07-10|08:39:23.298] logging to file system log dir=/opt/bscdata/erigon/logs file prefix=integration log level=info json=false INFO[07-10|08:39:23.299] [db] open label=chaindata sizeLimit=12TB pageSize=16384 panic: compressed file "v1-000000-000500-headers.seg" dictionary is corrupted: size 85.6 MB but no words in it, fileName: v1-000000-000500-headers.seg

goroutine 1 [running]: github.com/ledgerwatch/erigon/cmd/integration/commands.allSnapshots.func1() github.com/ledgerwatch/erigon/cmd/integration/commands/stages.go:1553 +0x450 sync.(Once).doSlow(0x26?, 0xc001b0f9b0?) sync/once.go:74 +0xbf sync.(Once).Do(...) sync/once.go:65 github.com/ledgerwatch/erigon/cmd/integration/commands.allSnapshots({0x273de30?, 0xc0025c8e10?}, {0x2745d60?, 0xc0000b9d00?}, {0x2749790?, 0xc0007ea3a0?}) github.com/ledgerwatch/erigon/cmd/integration/commands/stages.go:1528 +0x90 github.com/ledgerwatch/erigon/cmd/integration/commands.stageHeaders({0x274a208?, 0xc0000b9d00}, {0x273de30?, 0xc0025c8e10}, {0x2749790, 0xc0007ea3a0}) github.com/ledgerwatch/erigon/cmd/integration/commands/stages.go:742 +0x12d github.com/ledgerwatch/erigon/cmd/integration/commands.glob..func11(0x35f4160, {0x1e1dd4b?, 0x4?, 0x1e1dbe3?}) github.com/ledgerwatch/erigon/cmd/integration/commands/stages.go:99 +0x212 github.com/spf13/cobra.(Command).execute(0x35f4160, {0xc0025d5620, 0x3, 0x3}) github.com/spf13/cobra@v1.8.0/command.go:987 +0xaa3 github.com/spf13/cobra.(Command).ExecuteC(0x35f1ee0) github.com/spf13/cobra@v1.8.0/command.go:1115 +0x3ff github.com/spf13/cobra.(Command).Execute(...) github.com/spf13/cobra@v1.8.0/command.go:1039 github.com/spf13/cobra.(Command).ExecuteContext(0x4873bc?, {0x273de30?, 0xc0025c8e10?}) github.com/spf13/cobra@v1.8.0/command.go:1032 +0x47 main.main() github.com/ledgerwatch/erigon/cmd/integration/main.go:18 +0xe6

WangXiaopai595 commented 2 weeks ago

One way is download snapshots https://github.com/bnb-chain/bsc-snapshots?tab=readme-ov-file#source-3-erigon-archive-node8tb.

Is there no other solution? Due to the control of Chinese public security, I am unable to download snapshots in my region

zhy827827 commented 2 weeks ago

I just downloaded the snapshot and ran the new node again. The result is that it can't run! No matter what version is used！

zhy827827 commented 2 weeks ago

@WangXiaopai595 没用的，我就是部署一个新节点，不管用什么版本 都有问题 都panic！

WangXiaopai595 commented 2 weeks ago

@WangXiaopai595 没用的，我就是部署一个新节点，不管用什么版本 都有问题 都panic！

吓我一跳，如果真要重新下载快照，我这几个节点就废了，我所在地区这个下载IP和域名被公安拦截了，如果走vpn时间成本太大

blxdyx commented 2 weeks ago

One way is download snapshots https://github.com/bnb-chain/bsc-snapshots?tab=readme-ov-file#source-3-erigon-archive-node8tb.

Is there no other solution? Due to the control of Chinese public security, I am unable to download snapshots in my region

I am trying reset the trie, need hours ./integration stage_hash_state --chain=bsc --datadir=./data --reset ./integration stage_trie --chain=bsc --datadir=./data --reset

zhy827827 commented 2 weeks ago

One way is download snapshots https://github.com/bnb-chain/bsc-snapshots?tab=readme-ov-file#source-3-erigon-archive-node8tb.

Is there no other solution? Due to the control of Chinese public security, I am unable to download snapshots in my region

I am trying reset the trie, need hours ./integration stage_hash_state --chain=bsc --datadir=./data --reset ./integration stage_trie --chain=bsc --datadir=./data --reset

One of our nodes has been reporting Panic errors since using -- reset

blxdyx commented 2 weeks ago

One way is download snapshots https://github.com/bnb-chain/bsc-snapshots?tab=readme-ov-file#source-3-erigon-archive-node8tb.

Is there no other solution? Due to the control of Chinese public security, I am unable to download snapshots in my region

I am trying reset the trie, need hours ./integration stage_hash_state --chain=bsc --datadir=./data --reset ./integration stage_trie --chain=bsc --datadir=./data --reset

One of our nodes has been reporting Panic errors since using -- reset

The same command?

saravadeanil commented 2 weeks ago

@blxdyx Can you please help with this? 🙇

https://github.com/node-real/bsc-erigon/issues/430#issuecomment-2219867921

blxdyx commented 2 weeks ago

You unwind in wrong order. Must unwind state_stages before unwind headers. But actually it won't help with this.

temuera commented 2 weeks ago

@WangXiaopai595 weixin lai yi fa? wo de 78085

cshintov commented 2 weeks ago

https://github.com/node-real/bsc-erigon/issues/430#issuecomment-2219867921 @saravadeanil

are you sure unwinding 20 will work? @blxdyx why 20? if the upgrade was done prior to the unwind will it work if we unwind say a 100 or 1000?

saravadeanil commented 2 weeks ago

are you sure unwinding 20 will work? @blxdyx why 20? if the upgrade was done prior to the unwind will it work if we unwind say a 100 or 1000?

I can try this. Thanks 🙏

blxdyx commented 2 weeks ago

are you sure unwinding 20 will work? @blxdyx why 20? if the upgrade was done prior to the unwind will it work if we unwind say a 100 or 1000?

I can try this. Thanks 🙏

Actually it won't work. I am trying this, it will take hours.

./integration stage_hash_state --chain=bsc --datadir=./data --reset 
./integration stage_trie --chain=bsc --datadir=./data --reset
zhy827827 commented 2 weeks ago

i download snapshots https://github.com/bnb-chain/bsc-snapshots?tab=readme-ov-file#source-3-erigon-archive-node8tb.

Error occurs after sync the snapshot:

err="[1/15 Snapshots] compressed file \"v1-000000-000500-headers.seg\" dictionary is corrupted: size 85.6 MB but no words in it, fileName: v1-000000-000500-headers.seg"

I tried deleting the snapshot directory and the download directory.

I really can't do it anymore,how to fix this issues? @blxdyx

blxdyx commented 2 weeks ago

i download snapshots https://github.com/bnb-chain/bsc-snapshots?tab=readme-ov-file#source-3-erigon-archive-node8tb.

Error occurs after sync the snapshot:

err="[1/15 Snapshots] compressed file \"v1-000000-000500-headers.seg\" dictionary is corrupted: size 85.6 MB but no words in it, fileName: v1-000000-000500-headers.seg"

I tried deleting the snapshot directory and the download directory.

I really can't do it anymore,how to fix this issues? @blxdyx

Which version you use. And have you check the checksum? Never reproduce this problem.
You could add flag --no-downloader when you start node.

zhy827827 commented 2 weeks ago

i download snapshots https://github.com/bnb-chain/bsc-snapshots?tab=readme-ov-file#source-3-erigon-archive-node8tb. Error occurs after sync the snapshot: [1/15 Snapshots] can't build missed indices: v1-000000-000500-bodies.seg: can't index v1-000000-000500-bodies.seg: index panic: at=v1-000000-000500-bodies.seg, runtime error: slice bounds out of range [:-1], [type.go:377 panic.go:920 panic.go:140 decompress.go:656 type.go:418 block_types.go:111 type.go:73 type.go:235 type.go:355 block_snapshots.go:786 errgroup.go:78 asm_amd64.s:1650] I tried deleting the snapshot directory and the download directory. I really can't do it anymore,how to fix this issues? @blxdyx

Which version you use. And have you check the checksum?

erigon version 1.2.11-dev\erigon version 1.2.10-dev
I have tried all the above versions and deleted the directory three times

erigon version 1.2.12-dev I am still testing this version

cshintov commented 2 weeks ago

What happens if we don't rewind at all after the upgrade?

I was on 1.2.10. Now running 1.2.12!

blxdyx commented 2 weeks ago

What happens if we don't rewind at all after the upgrade?

I was on 1.2.10. Now running 1.2.12!

1.2.10 have same problem? This issue occur in v1.2.11.

cshintov commented 2 weeks ago

image

Are you sure only in 1.2.11? The release note says otherwise.

My nodes are running without any issues after upgrade from 1.2.10 to 1.2.12!

So I don't have to rewind?

blxdyx commented 2 weeks ago

Yes, i changed that now. It should don't have any problem in v1.2.10

cshintov commented 2 weeks ago

Cool. Thx 👍🏽

WangXiaopai595 commented 2 weeks ago

state_stages cannot be executed either

integration state_stages --unwind=20 --chain=bsc --datadir ./data/
INFO[07-10|18:15:23.144] logging to file system                   log dir=data/logs file prefix=integration log level=info json=false
INFO[07-10|18:15:23.159] [db] open                                label=chaindata sizeLimit=10TB pageSize=16384
INFO[07-10|18:15:27.701] [snapshots:all] Stat                     blocks=30500k indices=30500k alloc=2.4GB sys=2.6GB
INFO[07-10|18:15:27.704] [snapshots:all] Stat                     blocks=0k indices=0k alloc=2.4GB sys=2.6GB
INFO[07-10|18:15:27.736] Opening Database                         label=parlia path=/home/erigon/data/parlia
mdbx_setup_dxb:15946 filesize mismatch (expect 98295808b/23998p, have 100663296b/24576p)
mdbx_setup_dxb:16114 opening after an unclean shutdown, but boot-id(bf9c1e97497ae15e-3656c57cc4894668) is MATCH: rollback NOT needed, steady-sync NEEDED
INFO[07-10|18:15:27.746] Opening Database                         label=blob path=/home/erigon/data/blobs/blob
INFO[07-10|18:15:50.412] [10/15 CallTraces] Unwind                from=40343181 to=40343161
INFO[07-10|18:15:56.883] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=true
INFO[07-10|18:15:58.103] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=false
INFO[07-10|18:16:01.077] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=true codes=false
INFO[07-10|18:16:07.310] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=AccountChangeSet
INFO[07-10|18:16:10.050] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=StorageChangeSet
EROR[07-10|18:16:34.690] [9/15 IntermediateHashes] wrong trie root: 5ad231002c0f58070379de4ab617b7f0d4bb2f745217060f10b7e65b9eae1816, expected (from header): 7ffd370e413891d2e0d6e43d7d3a2aef0cf86772eafab24271b2f7d9c83ce882
saravadeanil commented 2 weeks ago

@WangXiaopai595 I am facing exactly same issue +1

zfy0701 commented 2 weeks ago

state_stages cannot be executed either

integration state_stages --unwind=20 --chain=bsc --datadir ./data/
INFO[07-10|18:15:23.144] logging to file system                   log dir=data/logs file prefix=integration log level=info json=false
INFO[07-10|18:15:23.159] [db] open                                label=chaindata sizeLimit=10TB pageSize=16384
INFO[07-10|18:15:27.701] [snapshots:all] Stat                     blocks=30500k indices=30500k alloc=2.4GB sys=2.6GB
INFO[07-10|18:15:27.704] [snapshots:all] Stat                     blocks=0k indices=0k alloc=2.4GB sys=2.6GB
INFO[07-10|18:15:27.736] Opening Database                         label=parlia path=/home/erigon/data/parlia
mdbx_setup_dxb:15946 filesize mismatch (expect 98295808b/23998p, have 100663296b/24576p)
mdbx_setup_dxb:16114 opening after an unclean shutdown, but boot-id(bf9c1e97497ae15e-3656c57cc4894668) is MATCH: rollback NOT needed, steady-sync NEEDED
INFO[07-10|18:15:27.746] Opening Database                         label=blob path=/home/erigon/data/blobs/blob
INFO[07-10|18:15:50.412] [10/15 CallTraces] Unwind                from=40343181 to=40343161
INFO[07-10|18:15:56.883] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=true
INFO[07-10|18:15:58.103] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=false
INFO[07-10|18:16:01.077] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=true codes=false
INFO[07-10|18:16:07.310] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=AccountChangeSet
INFO[07-10|18:16:10.050] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=StorageChangeSet
EROR[07-10|18:16:34.690] [9/15 IntermediateHashes] wrong trie root: 5ad231002c0f58070379de4ab617b7f0d4bb2f745217060f10b7e65b9eae1816, expected (from header): 7ffd370e413891d2e0d6e43d7d3a2aef0cf86772eafab24271b2f7d9c83ce882

same issue here

insider89 commented 2 weeks ago

Same issue here

sokiaoba commented 2 weeks ago

Changing the subject a bit, is it possible to move up the timing for the next disk snapshot? I am in the same situation as mentioned above, and in the worst case where data recovery is not possible, I am considering recovery from the disk snapshot. However, using the latest snapshot (from 6/25) would require synchronization time.

temuera commented 2 weeks ago

./integration state_stages --unwind=20 --chain=bsc --datadir=./data

INFO[07-10|21:32:57.922] [10/15 CallTraces] Unwind                from=40343181 to=40343161
INFO[07-10|21:32:58.152] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=true
INFO[07-10|21:32:58.181] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=false
INFO[07-10|21:32:58.333] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=true codes=false
INFO[07-10|21:32:58.524] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=AccountChangeSet
INFO[07-10|21:32:58.567] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=StorageChangeSet
EROR[07-10|21:32:59.396] [9/15 IntermediateHashes] wrong trie root: 5ad231002c0f58070379de4ab617b7f0d4bb2f745217060f10b7e65b9eae1816, expected (from header): 7ffd370e413891d2e0d6e43d7d3a2aef0cf86772eafab24271b2f7d9c83ce882
blxdyx commented 2 weeks ago

./integration state_stages --unwind=20 --chain=bsc --datadir=./data

INFO[07-10|21:32:57.922] [10/15 CallTraces] Unwind                from=40343181 to=40343161
INFO[07-10|21:32:58.152] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=true
INFO[07-10|21:32:58.181] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=false
INFO[07-10|21:32:58.333] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=true codes=false
INFO[07-10|21:32:58.524] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=AccountChangeSet
INFO[07-10|21:32:58.567] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=StorageChangeSet
EROR[07-10|21:32:59.396] [9/15 IntermediateHashes] wrong trie root: 5ad231002c0f58070379de4ab617b7f0d4bb2f745217060f10b7e65b9eae1816, expected (from header): 7ffd370e413891d2e0d6e43d7d3a2aef0cf86772eafab24271b2f7d9c83ce882

show print_stage

temuera commented 2 weeks ago

@blxdyx ./integration print_stages --chain=bsc --datadir=./data

INFO[07-11|00:07:14.101] logging to file system                   log dir=data/logs file prefix=integration log level=info json=false
INFO[07-11|00:07:14.103] [db] open                                label=chaindata sizeLimit=12TB pageSize=16384
INFO[07-11|00:07:17.489] [snapshots:all] Stat                     blocks=40253k indices=40253k alloc=3.1GB sys=3.2GB
INFO[07-11|00:07:17.493] [snapshots:all] Stat                     blocks=0k indices=0k alloc=3.1GB sys=3.2GB
Note: prune_at doesn't mean 'all data before were deleted' - it just mean stage.Prune function were run to this block. Because 1 stage may prune multiple data types to different prune distance.

                                 stage_at        prune_at
Snapshots                        40343180        0
Headers                          40343161        0
BorHeimdall                      0               0
BlockHashes                      40343181        0
Bodies                           40343161        0
Senders                          40343181        0
Execution                        40343181        40343181
Translation                      0               0
HashState                        40343181        0
IntermediateHashes               40343181        40343181
AccountHistoryIndex              40343181        0
StorageHistoryIndex              40343181        0
LogIndex                         40343181        0
CallTraces                       40343181        40343181
TxLookup                         40343181        40253000
Finish                           40343181        0
--
prune distance: 

blocks.v2: true, blocks=40252999, segments=40252999, indices=40252999
blocks.bor.v2: segments=0, indices=0

history.v3: false,  idx steps: 0.00, lastBlockInSnap=0, TxNums_Index(0,0)

sequence: EthTx=6012387253, NonCanonicalTx=38520

in db: first header 40253000, last header 40343161, first body 40253000, last body 40343161
WangXiaopai595 commented 2 weeks ago

Has anyone solved this problem

blxdyx commented 2 weeks ago

@blxdyx ./integration print_stages --chain=bsc --datadir=./data

INFO[07-11|00:07:14.101] logging to file system                   log dir=data/logs file prefix=integration log level=info json=false
INFO[07-11|00:07:14.103] [db] open                                label=chaindata sizeLimit=12TB pageSize=16384
INFO[07-11|00:07:17.489] [snapshots:all] Stat                     blocks=40253k indices=40253k alloc=3.1GB sys=3.2GB
INFO[07-11|00:07:17.493] [snapshots:all] Stat                     blocks=0k indices=0k alloc=3.1GB sys=3.2GB
Note: prune_at doesn't mean 'all data before were deleted' - it just mean stage.Prune function were run to this block. Because 1 stage may prune multiple data types to different prune distance.

                                 stage_at        prune_at
Snapshots                        40343180        0
Headers                          40343161        0
BorHeimdall                      0               0
BlockHashes                      40343181        0
Bodies                           40343161        0
Senders                          40343181        0
Execution                        40343181        40343181
Translation                      0               0
HashState                        40343181        0
IntermediateHashes               40343181        40343181
AccountHistoryIndex              40343181        0
StorageHistoryIndex              40343181        0
LogIndex                         40343181        0
CallTraces                       40343181        40343181
TxLookup                         40343181        40253000
Finish                           40343181        0
--
prune distance: 

blocks.v2: true, blocks=40252999, segments=40252999, indices=40252999
blocks.bor.v2: segments=0, indices=0

history.v3: false,  idx steps: 0.00, lastBlockInSnap=0, TxNums_Index(0,0)

sequence: EthTx=6012387253, NonCanonicalTx=38520

in db: first header 40253000, last header 40343161, first body 40253000, last body 40343161

Try to get the block 40343181. 

curl --location --request POST 'localhost:8545' --header 'content-type: application/json' --data-raw '[
{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["0x267968D",false],"id":1}]'
WangXiaopai595 commented 2 weeks ago

curl --location --request POST 'localhost:8545' --header 'content-type: application/json' --data-raw '[ {"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["0x267968D",false],"id":1}]'

{
    "jsonrpc": "2.0",
    "id": 1,
    "result": {
        "baseFeePerGas": "0x0",
        "blobGasUsed": "0x0",
        "difficulty": "0x2",
        "excessBlobGas": "0x0",
        "extraData": "0xd88301040a846765746888676f312e32312e35856c696e757800000048fa7b05f8b5831fffffb8608fc1dc6866d8ac258bc40a767e9e4a9239959d75689b8bcc15d59706539d49f2c08bcacc2bb0250adf2bf5aa5eefe937075442f07858ca24a0554ca3706cf6b6a9fb405992e97814545cb7276700bd94392eb69e6f57cdf48d34cb38cadcfb18f84c840267968ba01d26dcd578fa75c274c99df0f650f4d5af01a4851fe2c7be78901bf79bda8471840267968ca0b4abb99ee61f474ecbf94de57adffb1359871852055797e2394bcb352b52ce9480e3c25f287cea7da4f26c94210ec66a5a5e67fdf22286a5383eac614ce29d6aeb025fea7c26d2af2458f1047c2e7d0e2760b921c57881a3bfc0beeaea66a39ede01",
        "gasLimit": "0x85832a7",
        "gasUsed": "0x9f25bc",
        "hash": "0xd0420f6083a3aefd6f61ce6d70f4e57ffba85181c0eedbb7058e7e19a745b513",
        "logsBloom": "0x4532a255031414b060e20b4188552503ac98765909aa6b662822ae8cca8c809086341c7dd1141279942b8472203ea98c9d4092a96f8a644829305c2082b48db4776854480e485a58f139852f598ccab127911021c6f4ed00071604269c8460e4806ee6ee031a971b001c51c081129d3b18b031d90e937db14a07d4b48199ca96ca36f250183b3e00538b4e9000400bdd29651e05fb74082b155291d15942146d8a37162498040b24fe071b0d5706ebc42b378756f0c90589404f1889bc485834a8b1fe1a66424a42939312932501d7202499724044b496100e5a7a27b20de82852b2803e2050408703442404d1d50904b766ff271c1268e8f28931f818c654e8",
        "miner": "0xd3b0d838ccceae7ebf1781d11d1bb741db7fe1a7",
        "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
        "nonce": "0x0000000000000000",
        "number": "0x267968d",
        "parentHash": "0xb4abb99ee61f474ecbf94de57adffb1359871852055797e2394bcb352b52ce94",
        "receiptsRoot": "0x551079cfea24df5a848572338506b60d8082719227a4ac346adf86f68d5c9594",
        "sha3Uncles": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
        "size": "0x612f",
        "stateRoot": "0xf4104516aa47e17c61adc69033938cef5302a692d5f98199c0df94d5646d3008",
        "timestamp": "0x668e24e5",
        "totalDifficulty": "0x66f47e",
        "transactions": [
            "0xcc9121969e413b67c3072d768789424153db6c84199715ffe90b9029a714f1bd",
            "0x6645981d2730e48a93f3c3d39e7ccb2a97eb3c0225ccc247ed8c47dd8aeaaf4d",
            "0x0308d314c916e3536e53436031f0d6f1483774bdceef35b442bfc9a07e322f2b",
            "0x261bd9a9fe717be63c59bbd66bd1adf197129a94bedc7a7adfc4b1e65c9dcde7",
            "0x337dcd6777647f49baf5cb8f03915d4bebd7953f70f9e590bb433a4b21dbfa63",
            "0x1861e1d53ed018236f4e2b06fbe29cc54f31da7c399592e173c6e269461f1858",
            "0xa31c61b1e66e5ecd31dcdadf66c49f73f2c091b1c428c257a6b58a9a7380b5e6",
            "0x56dcbe3c3aa9b5b267a5769c3c335958f1ea7ffe71464e38a3571afa362f9380",
            "0xbe59c8b52d07a6189350acd54973eab4049c998b4ee17dc0d29151613bf6b741",
            "0x94596f293db12caf7d80246102544f6e2c88f534b4394ac1346d68a365e413d7",
            "0xd7648cccc5bfc171d5d107c13cae8c4216673246c2664eb7bfb9594cdffba99f",
            "0x84d3b6d9c675b476f081c5e2727d97a57335fc75c465e1af98d847ce0fa8c5b6",
            "0xb186178a3412e3429e792d1e115a054d93624f47bc90e007e2b2fcd3631cad2b",
            "0xd9bc56b924048ce12b7963bbe67c6d5b8092018a259cbb5a1f6c35e0972f2819",
            "0x98e1719bfbad019f32f5a0c6072585379242ee8245a1e5259a69290fcf831815",
            "0xbf5b2942b3e0f6d75a2f362b28087936baa30c68dbaf47b38a242d4ca9415f0f",
            "0x7975b9874dde2a944192b5d4501178d3ad81a39deb1201323e50356c86895531",
            "0x28d7d6219e46a29adad2717cee8bc4a8d5a72dcf4d9fbb5c92b17e1d717353ab",
            "0xb9f797a9f1933f3226efae53071f95874c8dd6bad69edc6e52ac26c723b789dd",
            "0x0e0f9c8343c4374d43e1f535a24515541fabf4d2447ab0b23dbc753a712249c7",
            "0xf131fca6627b69f34d1cfc169d037b51c78180b6bab690b3a6dfc96ac1c95fdf",
            "0x81429cc62aa73c860fe92986df0bbe88e2e08cbb703aaea51cf229fc581d1569",
            "0xbb35d7628bf183b660ab60cbbaf2498d5320e6c838d6a9cba37f8f7458f2afdb",
            "0xd55dd17fd1d131cb102785c8a66de6a2554fdd8ac26ab9888a30472888da1fff",
            "0x5a932335aef115520dc6473117a8433109a9aa871e341a5acb5be3aefd5894ce",
            "0x3c2cbeec9d6211003a69ef29f69fa0006be578c6e52712f5d781f142d2ce0153",
            "0x5f52064a41fb4350b110ffa14ac8a1cc31be324cbf5b63c22bd7f07302664e9b",
            "0x1e308ab2de8fa54742853df181e6d33f9755ba0395f984513defae1bc7222951",
            "0x2c769a0a1ab9607bb897f6109ad809296129577a2424cb3cbf0746d294d8135b",
            "0x89a43c4278457190359d350b6d5869c34fc9e8fc1c5001886fba28e2ec263a7e",
            "0x0917cfa6a203d033fdcf0ca2b8d0464b8c5fef38b4c75ef6f797be366f50bfbe",
            "0x3d60e22fa57b7746593785cab774fba3ea879f7a16c372cd656f99638fe649a0",
            "0x36f2c9849fe62f9c460bac49dfba8628191fb88f9e43cbf0cbd45f16a44d1be0",
            "0xc45d5e20d2932613f27e4482f084ab130d2a5690f9fea77a3f338af704e93370",
            "0xd3c4101a2db75f0cef66012587af5830da9dfec071dd15b78e15e49b5e78bda1",
            "0x9d53057a9ffa85b14fa21a7dfd841b8fe596dcaa89cf86f6b6ee6f5cb1f8f743",
            "0x1f3fac712d6285885075ba99c40ba9a9ee2450a5060debc273b6ba1bc358ecf2",
            "0x466a9138a6bc780f0d7d45b97ccdef32fd1252789fc98c1a92916c3401fb2fc2",
            "0x775768dfb635f93a3806b8a106b8b3f7ad2d5afc3fd843b4ca75d14e36361df8",
            "0x3089d309fd06482c9bf3d122d9c240a0800a0f4b685be6081f22a2fae269062b",
            "0x7fc73e3c15ec4c5d46b9698563d852ff8edee13f89f551a6618dd095bd6c7c40",
            "0x6c4db8bcf43fb39308508225e2bf28b88675710d5968ae7d151a63505f10020b",
            "0xdf69dffde2c8e2db6b7cb98514fd7792d0c556d32c486780c606b54b98ed0404",
            "0xb943f0460aec31165ef6a8b7ceb3a5b16b758c1076eeb1eee7dff71f5977fcb3",
            "0xdd18d2c8713cd5cd460d5066459a336bc259312f8a0a92361ef6468c66adbc07",
            "0x21b89d601a07f7f590e67dc4e53203daf47606bc516d91ce5018c61b0c638483",
            "0x18b21ec76a03cd9428874a85b3fb425a2868fd5f01887c060d1c3ed1ae8a8c7e",
            "0xa5a42947998a88ca57bc2d455baad584daea49298a5a641f22a4df02ae1bd42c",
            "0x02c605286e6717e52f3bf85e70c4524ee398bb206f56a601be8dd0973c34be63",
            "0x38a9bcd8c78bd66a93b7b836051b22942040ff192616dc13dab58915ad2adb40",
            "0x7fc8d3edd112aa868e865da0b2c80c9bbae94079e3874bc49de0f0e5a8761a35",
            "0xc39d0dfaf1efd7aa9ff26061ef061ca95b42e2102c5fe600cae61c7ef4bb5c32",
            "0x177bf436d3cd5ed30175389defc469ee0aa2fef0ee2e7b2122c7d2160af58547",
            "0xfef90032047ad3f1ceaf598540452060421f03c47f9f5e149b1e0188a3997751",
            "0x1415ee7a6b921fcc6fd2b9e3dad0b82659f02893debd6b238b8f2dbba35c7099",
            "0xba558c9c6d30220064156ee7632971cd0659518dbb7feacea837dcfba382b7cb",
            "0xc9a41027b7b089686c2c0f7b7eb6bab3273b5682313ce29e31f558264183aa9e",
            "0x2a0f7ce45eeff49c7dec2564c7c287e52bcfc13ac6dd53da88b8ce566ef2bf5c",
            "0x8795b391f7ad29787074c71b9b4d9fd6bf38338ad650961388c5d9257e14e7e4",
            "0xaaa36ce6b1bec4a6d3e0afd721477d735c80a0f355bd545e374f5dd8aa11efb5",
            "0x0d8bf97c3500a94bf3f94f91d64cac836ac3e79426ca639fbc4b0ce5063b31bd",
            "0xbc2ef2a771a49b4337906a3d96062b48240a16d526ff1017b3718ec50704128d",
            "0x210d18465ba2c6f6b78b903a66ccdc71fe2afd86175b7f7155ede7844c767ffe",
            "0x14a19a8fa40c6931b133eeba056695e8926a3fb05f571f74ff07a3b84b02e180",
            "0x4877164756dde6a97f06237da294a72ae33ffb13a4285dcdde15de63d3060512",
            "0xb1964903e09160a8d59f8edda7272d9f1bfe959ba279b685bd6edc001862076d",
            "0x02f0a30143f4e31d8bd609611c556cabbd3c989adefbc78272a664a780467852",
            "0x899f401acbdcdd0d1af85f323ea7ab1f937ad282cd9456c211a8e803c0f506c6",
            "0xbb0bfa6b8fcee6b767b3ca80ed6d6f9df3eacff383dd866b37d83adea71284f3",
            "0x7d40c877a032d0f4176877ae27710155d476c65bee64030a6ae4206c805c1a7d",
            "0xad84fccb1fd54b21a22c594d64f98c4fd0ca404d7a640c20edb5d322114ab90b",
            "0x4fe46570f318e06463399aba1463a536bc853d4f661c15289dcc03f8b3c66234",
            "0xf2a67c28bb12d1611c39289f82a3dfab4cf1747e5c50efe5b69e26d36493a318",
            "0xad13bdb7361e62ed7d8cfc38df41f6a2bc2fff9dd8d287779744dc9a033617c0",
            "0x10d950533ffe4c56f137988d5f5775c7bc99ea69c738a76d72058952d95199a5",
            "0x7b045828e0245295c27507d3e729729a660a1dce7b32ee6c345919a429c37317",
            "0xb93166afd7093f0a73e95f88b3648a9de9a5364671b96aaa4ee1e869766b54e0",
            "0x1b7d477e9f20818aa596dbf43e6c5591285517e55950bb5cb56b227c0e840d0f"
        ],
        "transactionsRoot": "0x59ec812ef69c4e683c9173dc41b03f31ce883eab9b80260779f2e7723e2065ea",
        "uncles": [],
        "withdrawals": [],
        "withdrawalsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421"
    }
}
blxdyx commented 2 weeks ago

curl --location --request POST 'localhost:8545' --header 'content-type: application/json' --data-raw '[ {"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["0x267968D",false],"id":1}]'

{
    "jsonrpc": "2.0",
    "id": 1,
    "result": {
        "baseFeePerGas": "0x0",
        "blobGasUsed": "0x0",
        "difficulty": "0x2",
        "excessBlobGas": "0x0",
        "extraData": "0xd88301040a846765746888676f312e32312e35856c696e757800000048fa7b05f8b5831fffffb8608fc1dc6866d8ac258bc40a767e9e4a9239959d75689b8bcc15d59706539d49f2c08bcacc2bb0250adf2bf5aa5eefe937075442f07858ca24a0554ca3706cf6b6a9fb405992e97814545cb7276700bd94392eb69e6f57cdf48d34cb38cadcfb18f84c840267968ba01d26dcd578fa75c274c99df0f650f4d5af01a4851fe2c7be78901bf79bda8471840267968ca0b4abb99ee61f474ecbf94de57adffb1359871852055797e2394bcb352b52ce9480e3c25f287cea7da4f26c94210ec66a5a5e67fdf22286a5383eac614ce29d6aeb025fea7c26d2af2458f1047c2e7d0e2760b921c57881a3bfc0beeaea66a39ede01",
        "gasLimit": "0x85832a7",
        "gasUsed": "0x9f25bc",
        "hash": "0xd0420f6083a3aefd6f61ce6d70f4e57ffba85181c0eedbb7058e7e19a745b513",
        "logsBloom": "0x4532a255031414b060e20b4188552503ac98765909aa6b662822ae8cca8c809086341c7dd1141279942b8472203ea98c9d4092a96f8a644829305c2082b48db4776854480e485a58f139852f598ccab127911021c6f4ed00071604269c8460e4806ee6ee031a971b001c51c081129d3b18b031d90e937db14a07d4b48199ca96ca36f250183b3e00538b4e9000400bdd29651e05fb74082b155291d15942146d8a37162498040b24fe071b0d5706ebc42b378756f0c90589404f1889bc485834a8b1fe1a66424a42939312932501d7202499724044b496100e5a7a27b20de82852b2803e2050408703442404d1d50904b766ff271c1268e8f28931f818c654e8",
        "miner": "0xd3b0d838ccceae7ebf1781d11d1bb741db7fe1a7",
        "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
        "nonce": "0x0000000000000000",
        "number": "0x267968d",
        "parentHash": "0xb4abb99ee61f474ecbf94de57adffb1359871852055797e2394bcb352b52ce94",
        "receiptsRoot": "0x551079cfea24df5a848572338506b60d8082719227a4ac346adf86f68d5c9594",
        "sha3Uncles": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
        "size": "0x612f",
        "stateRoot": "0xf4104516aa47e17c61adc69033938cef5302a692d5f98199c0df94d5646d3008",
        "timestamp": "0x668e24e5",
        "totalDifficulty": "0x66f47e",
        "transactions": [
            "0xcc9121969e413b67c3072d768789424153db6c84199715ffe90b9029a714f1bd",
            "0x6645981d2730e48a93f3c3d39e7ccb2a97eb3c0225ccc247ed8c47dd8aeaaf4d",
            "0x0308d314c916e3536e53436031f0d6f1483774bdceef35b442bfc9a07e322f2b",
            "0x261bd9a9fe717be63c59bbd66bd1adf197129a94bedc7a7adfc4b1e65c9dcde7",
            "0x337dcd6777647f49baf5cb8f03915d4bebd7953f70f9e590bb433a4b21dbfa63",
            "0x1861e1d53ed018236f4e2b06fbe29cc54f31da7c399592e173c6e269461f1858",
            "0xa31c61b1e66e5ecd31dcdadf66c49f73f2c091b1c428c257a6b58a9a7380b5e6",
            "0x56dcbe3c3aa9b5b267a5769c3c335958f1ea7ffe71464e38a3571afa362f9380",
            "0xbe59c8b52d07a6189350acd54973eab4049c998b4ee17dc0d29151613bf6b741",
            "0x94596f293db12caf7d80246102544f6e2c88f534b4394ac1346d68a365e413d7",
            "0xd7648cccc5bfc171d5d107c13cae8c4216673246c2664eb7bfb9594cdffba99f",
            "0x84d3b6d9c675b476f081c5e2727d97a57335fc75c465e1af98d847ce0fa8c5b6",
            "0xb186178a3412e3429e792d1e115a054d93624f47bc90e007e2b2fcd3631cad2b",
            "0xd9bc56b924048ce12b7963bbe67c6d5b8092018a259cbb5a1f6c35e0972f2819",
            "0x98e1719bfbad019f32f5a0c6072585379242ee8245a1e5259a69290fcf831815",
            "0xbf5b2942b3e0f6d75a2f362b28087936baa30c68dbaf47b38a242d4ca9415f0f",
            "0x7975b9874dde2a944192b5d4501178d3ad81a39deb1201323e50356c86895531",
            "0x28d7d6219e46a29adad2717cee8bc4a8d5a72dcf4d9fbb5c92b17e1d717353ab",
            "0xb9f797a9f1933f3226efae53071f95874c8dd6bad69edc6e52ac26c723b789dd",
            "0x0e0f9c8343c4374d43e1f535a24515541fabf4d2447ab0b23dbc753a712249c7",
            "0xf131fca6627b69f34d1cfc169d037b51c78180b6bab690b3a6dfc96ac1c95fdf",
            "0x81429cc62aa73c860fe92986df0bbe88e2e08cbb703aaea51cf229fc581d1569",
            "0xbb35d7628bf183b660ab60cbbaf2498d5320e6c838d6a9cba37f8f7458f2afdb",
            "0xd55dd17fd1d131cb102785c8a66de6a2554fdd8ac26ab9888a30472888da1fff",
            "0x5a932335aef115520dc6473117a8433109a9aa871e341a5acb5be3aefd5894ce",
            "0x3c2cbeec9d6211003a69ef29f69fa0006be578c6e52712f5d781f142d2ce0153",
            "0x5f52064a41fb4350b110ffa14ac8a1cc31be324cbf5b63c22bd7f07302664e9b",
            "0x1e308ab2de8fa54742853df181e6d33f9755ba0395f984513defae1bc7222951",
            "0x2c769a0a1ab9607bb897f6109ad809296129577a2424cb3cbf0746d294d8135b",
            "0x89a43c4278457190359d350b6d5869c34fc9e8fc1c5001886fba28e2ec263a7e",
            "0x0917cfa6a203d033fdcf0ca2b8d0464b8c5fef38b4c75ef6f797be366f50bfbe",
            "0x3d60e22fa57b7746593785cab774fba3ea879f7a16c372cd656f99638fe649a0",
            "0x36f2c9849fe62f9c460bac49dfba8628191fb88f9e43cbf0cbd45f16a44d1be0",
            "0xc45d5e20d2932613f27e4482f084ab130d2a5690f9fea77a3f338af704e93370",
            "0xd3c4101a2db75f0cef66012587af5830da9dfec071dd15b78e15e49b5e78bda1",
            "0x9d53057a9ffa85b14fa21a7dfd841b8fe596dcaa89cf86f6b6ee6f5cb1f8f743",
            "0x1f3fac712d6285885075ba99c40ba9a9ee2450a5060debc273b6ba1bc358ecf2",
            "0x466a9138a6bc780f0d7d45b97ccdef32fd1252789fc98c1a92916c3401fb2fc2",
            "0x775768dfb635f93a3806b8a106b8b3f7ad2d5afc3fd843b4ca75d14e36361df8",
            "0x3089d309fd06482c9bf3d122d9c240a0800a0f4b685be6081f22a2fae269062b",
            "0x7fc73e3c15ec4c5d46b9698563d852ff8edee13f89f551a6618dd095bd6c7c40",
            "0x6c4db8bcf43fb39308508225e2bf28b88675710d5968ae7d151a63505f10020b",
            "0xdf69dffde2c8e2db6b7cb98514fd7792d0c556d32c486780c606b54b98ed0404",
            "0xb943f0460aec31165ef6a8b7ceb3a5b16b758c1076eeb1eee7dff71f5977fcb3",
            "0xdd18d2c8713cd5cd460d5066459a336bc259312f8a0a92361ef6468c66adbc07",
            "0x21b89d601a07f7f590e67dc4e53203daf47606bc516d91ce5018c61b0c638483",
            "0x18b21ec76a03cd9428874a85b3fb425a2868fd5f01887c060d1c3ed1ae8a8c7e",
            "0xa5a42947998a88ca57bc2d455baad584daea49298a5a641f22a4df02ae1bd42c",
            "0x02c605286e6717e52f3bf85e70c4524ee398bb206f56a601be8dd0973c34be63",
            "0x38a9bcd8c78bd66a93b7b836051b22942040ff192616dc13dab58915ad2adb40",
            "0x7fc8d3edd112aa868e865da0b2c80c9bbae94079e3874bc49de0f0e5a8761a35",
            "0xc39d0dfaf1efd7aa9ff26061ef061ca95b42e2102c5fe600cae61c7ef4bb5c32",
            "0x177bf436d3cd5ed30175389defc469ee0aa2fef0ee2e7b2122c7d2160af58547",
            "0xfef90032047ad3f1ceaf598540452060421f03c47f9f5e149b1e0188a3997751",
            "0x1415ee7a6b921fcc6fd2b9e3dad0b82659f02893debd6b238b8f2dbba35c7099",
            "0xba558c9c6d30220064156ee7632971cd0659518dbb7feacea837dcfba382b7cb",
            "0xc9a41027b7b089686c2c0f7b7eb6bab3273b5682313ce29e31f558264183aa9e",
            "0x2a0f7ce45eeff49c7dec2564c7c287e52bcfc13ac6dd53da88b8ce566ef2bf5c",
            "0x8795b391f7ad29787074c71b9b4d9fd6bf38338ad650961388c5d9257e14e7e4",
            "0xaaa36ce6b1bec4a6d3e0afd721477d735c80a0f355bd545e374f5dd8aa11efb5",
            "0x0d8bf97c3500a94bf3f94f91d64cac836ac3e79426ca639fbc4b0ce5063b31bd",
            "0xbc2ef2a771a49b4337906a3d96062b48240a16d526ff1017b3718ec50704128d",
            "0x210d18465ba2c6f6b78b903a66ccdc71fe2afd86175b7f7155ede7844c767ffe",
            "0x14a19a8fa40c6931b133eeba056695e8926a3fb05f571f74ff07a3b84b02e180",
            "0x4877164756dde6a97f06237da294a72ae33ffb13a4285dcdde15de63d3060512",
            "0xb1964903e09160a8d59f8edda7272d9f1bfe959ba279b685bd6edc001862076d",
            "0x02f0a30143f4e31d8bd609611c556cabbd3c989adefbc78272a664a780467852",
            "0x899f401acbdcdd0d1af85f323ea7ab1f937ad282cd9456c211a8e803c0f506c6",
            "0xbb0bfa6b8fcee6b767b3ca80ed6d6f9df3eacff383dd866b37d83adea71284f3",
            "0x7d40c877a032d0f4176877ae27710155d476c65bee64030a6ae4206c805c1a7d",
            "0xad84fccb1fd54b21a22c594d64f98c4fd0ca404d7a640c20edb5d322114ab90b",
            "0x4fe46570f318e06463399aba1463a536bc853d4f661c15289dcc03f8b3c66234",
            "0xf2a67c28bb12d1611c39289f82a3dfab4cf1747e5c50efe5b69e26d36493a318",
            "0xad13bdb7361e62ed7d8cfc38df41f6a2bc2fff9dd8d287779744dc9a033617c0",
            "0x10d950533ffe4c56f137988d5f5775c7bc99ea69c738a76d72058952d95199a5",
            "0x7b045828e0245295c27507d3e729729a660a1dce7b32ee6c345919a429c37317",
            "0xb93166afd7093f0a73e95f88b3648a9de9a5364671b96aaa4ee1e869766b54e0",
            "0x1b7d477e9f20818aa596dbf43e6c5591285517e55950bb5cb56b227c0e840d0f"
        ],
        "transactionsRoot": "0x59ec812ef69c4e683c9173dc41b03f31ce883eab9b80260779f2e7723e2065ea",
        "uncles": [],
        "withdrawals": [],
        "withdrawalsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421"
    }
}

Thanks, also show print stages. Also could you send me the logs, it will help me debug.

WangXiaopai595 commented 2 weeks ago

integration print_stages --chain=bsc --datadir=./data


integration print_stages --chain=bsc --datadir=./data 
INFO[07-11|10:10:34.860] logging to file system                   log dir=data/logs file prefix=integration log level=info json=false
INFO[07-11|10:10:34.864] [db] open                                label=chaindata sizeLimit=10TB pageSize=16384
INFO[07-11|10:10:40.180] [snapshots:all] Stat                     blocks=30500k indices=30500k alloc=2.4GB sys=2.5GB
INFO[07-11|10:10:40.184] [snapshots:all] Stat                     blocks=0k indices=0k alloc=2.4GB sys=2.5GB
Note: prune_at doesn't mean 'all data before were deleted' - it just mean stage.Prune function were run to this block. Because 1 stage may prune multiple data types to different prune distance.
             stage_at    prune_at

Snapshots 40343181 0 Headers 40367077 0 BorHeimdall 0 0 BlockHashes 40367077 0 Bodies 40343181 0 Senders 40343181 0 Execution 40343181 40343181 Translation 0 0 HashState 40343181 0 IntermediateHashes 40343181 40343181 AccountHistoryIndex 40343181 0 StorageHistoryIndex 40343181 0 LogIndex 40343181 0 CallTraces 40343181 40343181 TxLookup 40343181 33510000 Finish 40343181 0

prune distance:

blocks.v2: true, blocks=30499999, segments=30499999, indices=30499999 blocks.bor.v2: segments=0, indices=0

history.v3: false, idx steps: 0.00, lastBlockInSnap=0, TxNums_Index(0,0)

sequence: EthTx=6014262696, NonCanonicalTx=68345

in db: first header 31753500, last header 40367077, first body 31753500, last body 40349677

blxdyx commented 2 weeks ago

integration print_stages --chain=bsc --datadir=./data

integration print_stages --chain=bsc --datadir=./data 
INFO[07-11|10:10:34.860] logging to file system                   log dir=data/logs file prefix=integration log level=info json=false
INFO[07-11|10:10:34.864] [db] open                                label=chaindata sizeLimit=10TB pageSize=16384
INFO[07-11|10:10:40.180] [snapshots:all] Stat                     blocks=30500k indices=30500k alloc=2.4GB sys=2.5GB
INFO[07-11|10:10:40.184] [snapshots:all] Stat                     blocks=0k indices=0k alloc=2.4GB sys=2.5GB
Note: prune_at doesn't mean 'all data before were deleted' - it just mean stage.Prune function were run to this block. Because 1 stage may prune multiple data types to different prune distance.

               stage_at    prune_at
Snapshots              40343181    0
Headers            40367077    0
BorHeimdall            0       0
BlockHashes            40367077    0
Bodies                 40343181    0
Senders            40343181    0
Execution              40343181    40343181
Translation            0       0
HashState              40343181    0
IntermediateHashes         40343181    40343181
AccountHistoryIndex        40343181    0
StorageHistoryIndex        40343181    0
LogIndex           40343181    0
CallTraces             40343181    40343181
TxLookup           40343181    33510000
Finish                 40343181    0
--
prune distance: 

blocks.v2: true, blocks=30499999, segments=30499999, indices=30499999
blocks.bor.v2: segments=0, indices=0

history.v3: false,  idx steps: 0.00, lastBlockInSnap=0, TxNums_Index(0,0)

sequence: EthTx=6014262696, NonCanonicalTx=68345

in db: first header 31753500, last header 40367077, first body 31753500, last body 40349677

Make integration use the branch.https://github.com/blxdyx/bsc-erigon/tree/temp_fix And then show ./integration state_stages --unwind=20 --chain=bsc --datadir=./data

WangXiaopai595 commented 2 weeks ago

integration print_stages --chain=bsc --datadir=./data

integration print_stages --chain=bsc --datadir=./data 
INFO[07-11|10:10:34.860] logging to file system                   log dir=data/logs file prefix=integration log level=info json=false
INFO[07-11|10:10:34.864] [db] open                                label=chaindata sizeLimit=10TB pageSize=16384
INFO[07-11|10:10:40.180] [snapshots:all] Stat                     blocks=30500k indices=30500k alloc=2.4GB sys=2.5GB
INFO[07-11|10:10:40.184] [snapshots:all] Stat                     blocks=0k indices=0k alloc=2.4GB sys=2.5GB
Note: prune_at doesn't mean 'all data before were deleted' - it just mean stage.Prune function were run to this block. Because 1 stage may prune multiple data types to different prune distance.

                 stage_at    prune_at
Snapshots            40343181    0
Headers              40367077    0
BorHeimdall              0       0
BlockHashes              40367077    0
Bodies               40343181    0
Senders              40343181    0
Execution            40343181    40343181
Translation              0       0
HashState            40343181    0
IntermediateHashes       40343181    40343181
AccountHistoryIndex          40343181    0
StorageHistoryIndex          40343181    0
LogIndex             40343181    0
CallTraces           40343181    40343181
TxLookup             40343181    33510000
Finish               40343181    0
--
prune distance: 

blocks.v2: true, blocks=30499999, segments=30499999, indices=30499999
blocks.bor.v2: segments=0, indices=0

history.v3: false,  idx steps: 0.00, lastBlockInSnap=0, TxNums_Index(0,0)

sequence: EthTx=6014262696, NonCanonicalTx=68345

in db: first header 31753500, last header 40367077, first body 31753500, last body 40349677

Build integration from the branch.https://github.com/blxdyx/bsc-erigon/tree/temp_fix And then show ./integration state_stages --unwind=20 --chain=bsc --datadir=./data

integration state_stages --unwind=20 --chain=bsc --datadir=./data
INFO[07-11|10:14:04.959] logging to file system                   log dir=data/logs file prefix=integration log level=info json=false
INFO[07-11|10:14:04.970] [db] open                                label=chaindata sizeLimit=10TB pageSize=16384
INFO[07-11|10:14:09.636] [snapshots:all] Stat                     blocks=30500k indices=30500k alloc=2.3GB sys=2.5GB
INFO[07-11|10:14:09.639] [snapshots:all] Stat                     blocks=0k indices=0k alloc=2.3GB sys=2.5GB
INFO[07-11|10:14:09.667] Opening Database                         label=parlia path=/home/erigon/data/parlia
mdbx_setup_dxb:15946 filesize mismatch (expect 98529280b/24055p, have 100663296b/24576p)
mdbx_setup_dxb:16114 opening after an unclean shutdown, but boot-id(bf9c1e97497ae15e-3656c57cc4894668) is MATCH: rollback NOT needed, steady-sync NEEDED
INFO[07-11|10:14:09.675] Opening Database                         label=blob path=/home/erigon/data/blobs/blob
INFO[07-11|10:14:14.112] [10/15 CallTraces] Unwind                from=40343181 to=40343161
INFO[07-11|10:14:15.150] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=true
INFO[07-11|10:14:15.229] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=false
INFO[07-11|10:14:15.431] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=true codes=false
INFO[07-11|10:14:15.946] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=AccountChangeSet
INFO[07-11|10:14:16.057] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=StorageChangeSet
EROR[07-11|10:14:18.188] [9/15 IntermediateHashes] wrong trie root: 5ad231002c0f58070379de4ab617b7f0d4bb2f745217060f10b7e65b9eae1816, expected (from header): 7ffd370e413891d2e0d6e43d7d3a2aef0cf86772eafab24271b2f7d9c83ce882
blxdyx commented 2 weeks ago

integration print_stages --chain=bsc --datadir=./data

integration print_stages --chain=bsc --datadir=./data 
INFO[07-11|10:10:34.860] logging to file system                   log dir=data/logs file prefix=integration log level=info json=false
INFO[07-11|10:10:34.864] [db] open                                label=chaindata sizeLimit=10TB pageSize=16384
INFO[07-11|10:10:40.180] [snapshots:all] Stat                     blocks=30500k indices=30500k alloc=2.4GB sys=2.5GB
INFO[07-11|10:10:40.184] [snapshots:all] Stat                     blocks=0k indices=0k alloc=2.4GB sys=2.5GB
Note: prune_at doesn't mean 'all data before were deleted' - it just mean stage.Prune function were run to this block. Because 1 stage may prune multiple data types to different prune distance.

               stage_at    prune_at
Snapshots              40343181    0
Headers            40367077    0
BorHeimdall            0       0
BlockHashes            40367077    0
Bodies                 40343181    0
Senders            40343181    0
Execution              40343181    40343181
Translation            0       0
HashState              40343181    0
IntermediateHashes         40343181    40343181
AccountHistoryIndex        40343181    0
StorageHistoryIndex        40343181    0
LogIndex           40343181    0
CallTraces             40343181    40343181
TxLookup           40343181    33510000
Finish                 40343181    0
--
prune distance: 

blocks.v2: true, blocks=30499999, segments=30499999, indices=30499999
blocks.bor.v2: segments=0, indices=0

history.v3: false,  idx steps: 0.00, lastBlockInSnap=0, TxNums_Index(0,0)

sequence: EthTx=6014262696, NonCanonicalTx=68345

in db: first header 31753500, last header 40367077, first body 31753500, last body 40349677

Build integration from the branch.https://github.com/blxdyx/bsc-erigon/tree/temp_fix And then show ./integration state_stages --unwind=20 --chain=bsc --datadir=./data

integration state_stages --unwind=20 --chain=bsc --datadir=./data
INFO[07-11|10:14:04.959] logging to file system                   log dir=data/logs file prefix=integration log level=info json=false
INFO[07-11|10:14:04.970] [db] open                                label=chaindata sizeLimit=10TB pageSize=16384
INFO[07-11|10:14:09.636] [snapshots:all] Stat                     blocks=30500k indices=30500k alloc=2.3GB sys=2.5GB
INFO[07-11|10:14:09.639] [snapshots:all] Stat                     blocks=0k indices=0k alloc=2.3GB sys=2.5GB
INFO[07-11|10:14:09.667] Opening Database                         label=parlia path=/home/erigon/data/parlia
mdbx_setup_dxb:15946 filesize mismatch (expect 98529280b/24055p, have 100663296b/24576p)
mdbx_setup_dxb:16114 opening after an unclean shutdown, but boot-id(bf9c1e97497ae15e-3656c57cc4894668) is MATCH: rollback NOT needed, steady-sync NEEDED
INFO[07-11|10:14:09.675] Opening Database                         label=blob path=/home/erigon/data/blobs/blob
INFO[07-11|10:14:14.112] [10/15 CallTraces] Unwind                from=40343181 to=40343161
INFO[07-11|10:14:15.150] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=true
INFO[07-11|10:14:15.229] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=false
INFO[07-11|10:14:15.431] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=true codes=false
INFO[07-11|10:14:15.946] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=AccountChangeSet
INFO[07-11|10:14:16.057] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=StorageChangeSet
EROR[07-11|10:14:18.188] [9/15 IntermediateHashes] wrong trie root: 5ad231002c0f58070379de4ab617b7f0d4bb2f745217060f10b7e65b9eae1816, expected (from header): 7ffd370e413891d2e0d6e43d7d3a2aef0cf86772eafab24271b2f7d9c83ce882

Make integration use this https://github.com/blxdyx/bsc-erigon/tree/temp_fix

WangXiaopai595 commented 2 weeks ago

Make integration use this https://github.com/blxdyx/bsc-erigon/tree/temp_fix

/home/erigon/bsc-erigon-temp_fix/build/bin/integration state_stages --unwind=20 --chain=bsc --datadir=./data
INFO[07-11|10:21:42.116] logging to file system                   log dir=data/logs file prefix=integration log level=info json=false
mdbx_setup_dxb:15946 filesize mismatch (expect 8792955682816b/536679424p, have 8793844875264b/536733696p)
INFO[07-11|10:21:42.129] [db] open                                label=chaindata sizeLimit=10TB pageSize=16384
INFO[07-11|10:21:46.688] [snapshots:all] Stat                     blocks=30500k indices=30500k alloc=2.3GB sys=2.4GB
INFO[07-11|10:21:46.692] [snapshots:all] Stat                     blocks=0k indices=0k alloc=2.3GB sys=2.4GB
INFO[07-11|10:21:46.720] Opening Database                         label=parlia path=/home/erigon/data/parlia
mdbx_setup_dxb:16114 opening after an unclean shutdown, but boot-id(bf9c1e97497ae15e-3656c57cc4894668) is MATCH: rollback NOT needed, steady-sync NEEDED
INFO[07-11|10:21:46.728] Opening Database                         label=blob path=/home/erigon/data/blobs/blob
INFO[07-11|10:21:49.151] [10/15 CallTraces] Unwind                from=40343181 to=40343161
INFO[07-11|10:21:49.566] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=true
INFO[07-11|10:21:49.609] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=false
INFO[07-11|10:21:49.746] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=true codes=false
INFO[07-11|10:21:50.067] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=AccountChangeSet
INFO[07-11|10:21:50.134] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=StorageChangeSet
EROR[07-11|10:21:51.373] [9/15 IntermediateHashes] block Number 40343161, block hash 0xadd61ef1d94009f180c11f329ee0379c53e49d862c36439129f1d8820f6a7e2c, err = wrong trie root: 5ad231002c0f58070379de4ab617b7f0d4bb2f745217060f10b7e65b9eae1816, expected (from header): 7ffd370e413891d2e0d6e43d7d3a2aef0cf86772eafab24271b2f7d9c83ce882
blxdyx commented 2 weeks ago

Make integration use this https://github.com/blxdyx/bsc-erigon/tree/temp_fix

/home/erigon/bsc-erigon-temp_fix/build/bin/integration state_stages --unwind=20 --chain=bsc --datadir=./data
INFO[07-11|10:21:42.116] logging to file system                   log dir=data/logs file prefix=integration log level=info json=false
mdbx_setup_dxb:15946 filesize mismatch (expect 8792955682816b/536679424p, have 8793844875264b/536733696p)
INFO[07-11|10:21:42.129] [db] open                                label=chaindata sizeLimit=10TB pageSize=16384
INFO[07-11|10:21:46.688] [snapshots:all] Stat                     blocks=30500k indices=30500k alloc=2.3GB sys=2.4GB
INFO[07-11|10:21:46.692] [snapshots:all] Stat                     blocks=0k indices=0k alloc=2.3GB sys=2.4GB
INFO[07-11|10:21:46.720] Opening Database                         label=parlia path=/home/erigon/data/parlia
mdbx_setup_dxb:16114 opening after an unclean shutdown, but boot-id(bf9c1e97497ae15e-3656c57cc4894668) is MATCH: rollback NOT needed, steady-sync NEEDED
INFO[07-11|10:21:46.728] Opening Database                         label=blob path=/home/erigon/data/blobs/blob
INFO[07-11|10:21:49.151] [10/15 CallTraces] Unwind                from=40343181 to=40343161
INFO[07-11|10:21:49.566] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=true
INFO[07-11|10:21:49.609] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=false codes=false
INFO[07-11|10:21:49.746] [8/15 HashState] Unwinding started       from=40343181 to=40343161 storage=true codes=false
INFO[07-11|10:21:50.067] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=AccountChangeSet
INFO[07-11|10:21:50.134] [9/15 IntermediateHashes] Unwinding      from=40343181 to=40343161 csbucket=StorageChangeSet
EROR[07-11|10:21:51.373] [9/15 IntermediateHashes] block Number 40343161, block hash 0xadd61ef1d94009f180c11f329ee0379c53e49d862c36439129f1d8820f6a7e2c, err = wrong trie root: 5ad231002c0f58070379de4ab617b7f0d4bb2f745217060f10b7e65b9eae1816, expected (from header): 7ffd370e413891d2e0d6e43d7d3a2aef0cf86772eafab24271b2f7d9c83ce882

Could you show me the logs about the node. Main about when insert the wrong 40343181 block.

WangXiaopai595 commented 2 weeks ago

Could you show me the logs about the node. Main about when insert the wrong 40343181 block. bsc.log

WangXiaopai595 commented 2 weeks ago

Could you show me the logs about the node. Main about when insert the wrong 40343181 block. bsc.log

This is the log after I restarted the node