search

okx / exchain

⛓️ EVM & Wasm $ IBC-compatible, OKTC is a L1 blockchain network built on top of Cosmos SDK that aims for optimal interoperability and performance ✨
https://www.okx.com/oktc
Other
575 stars 184 forks source link

Testnet 同步出错 #2706

Closed ghost closed 2 years ago

ghost commented 2 years ago

1. 详细描述

OKC Testnet 使用 https://okg-pub-hk.oss-cn-hongkong.aliyuncs.com/cdn/oec/snapshot/testnet-s0-fss-20221026-15749371-rocksdb.tar.gz 快照无法正常同步。刚开始可以同步区块，但是一旦同步到 Height<15757658> 就会出现

I[2022-10-31|06:36:17.048][4250] Height<15757658>, Tx<0>, BlockSize<2108>, RunTx<0ms>, . module=main
panic: Failed to process committed block (15757658:B5AD55C6A29388DE836C8A94FA793818DDF46ED307C96DFF1717A1A95BC1C02D): wrong Block.Header.AppHash.  Expected ABE2F56890E8ABAE8EC4993F15A1403ED8DF804194AEB4FF29D7303DC1250147, got 1582F39D94CE467FD3936B2A4AD6D28F958F926EBD0F3C22610AC8E250C7A405

2. 硬件配置

AWS EC2 r6i.large, 2 core 16 GB

3. exchaind版本号

尝试过 release 里面的 v1.6.4, v1.6.3, v1.6.2，均为相同错误。

现在版本是 ``` name: exchain server_name: exchaind client_name: exchaincli version: v1.6.4 commit: 8bf520469c4cab294eb48a36639bef2094788ebe build_tags: netgo rocksdb go: go version go1.18.6 linux/amd64 build_deps: - github.com/99designs/keyring@v1.1.6 - github.com/ChainSafe/go-schnorrkel@v0.0.0-20200405005733-88cbf1b4c40d - github.com/CosmWasm/wasmvm@v1.0.0 - github.com/VictoriaMetrics/fastcache@v1.8.0 - github.com/Workiva/go-datastructures@v1.0.52 - github.com/aliyun/alibaba-cloud-sdk-go@v1.61.18 - github.com/aliyun/aliyun-oss-go-sdk@v2.1.6+incompatible - github.com/apolloconfig/agollo/v4@v4.0.8 - github.com/bartekn/go-bip39@v0.0.0-20171116152956-a05967ea095d - github.com/beorn7/perks@v1.0.1 - github.com/bgentry/speakeasy@v0.1.0 - github.com/btcsuite/btcd@v0.21.0-beta - github.com/btcsuite/btcutil@v1.0.3-0.20201208143702-a53e38424cce - github.com/buger/jsonparser@v0.0.0-20181115193947-bf1c66bbce23 => github.com/buger/jsonparser@v1.0.0 - github.com/cespare/xxhash/v2@v2.1.2 - github.com/confio/ics23/go@v0.7.0 => github.com/cosmos/cosmos-sdk/ics23/go@v0.8.0 - github.com/cosmos/go-bip39@v0.0.0-20180819234021-555e2067c45d - github.com/cosmos/gorocksdb@v1.2.0 - github.com/davecgh/go-spew@v1.1.1 - github.com/deckarep/golang-set@v0.0.0-20180603214616-504e848d77ea - github.com/dgryski/go-rendezvous@v0.0.0-20200823014737-9f7001d12a5f - github.com/dvsekhvalnov/jose2go@v0.0.0-20200901110807-248326c1351b - github.com/edsrzf/mmap-go@v1.0.0 - github.com/enigmampc/btcutil@v1.0.3-0.20200723161021-e2fb6adb2a25 - github.com/ethereum/go-ethereum@v1.10.8 => github.com/okex/go-ethereum@v1.10.8-okc1 - github.com/fsnotify/fsnotify@v1.4.9 - github.com/gballet/go-libpcsclite@v0.0.0-20190607065134-2772fd86a8ff - github.com/go-errors/errors@v1.0.1 - github.com/go-kit/kit@v0.10.0 - github.com/go-logfmt/logfmt@v0.5.0 - github.com/go-redis/redis/v8@v8.11.4 - github.com/go-sql-driver/mysql@v1.6.0 - github.com/go-stack/stack@v1.8.0 - github.com/godbus/dbus@v0.0.0-20190726142602-4481cbc300e2 - github.com/gogo/gateway@v1.1.0 - github.com/gogo/protobuf@v1.3.2 => github.com/regen-network/protobuf@v1.3.3-alpha.regen.1 - github.com/golang/protobuf@v1.5.2 - github.com/golang/snappy@v0.0.4 - github.com/google/btree@v1.0.0 - github.com/google/gofuzz@v1.1.1-0.20200604201612-c04b05f3adfa - github.com/google/gops@v0.3.23 - github.com/google/uuid@v1.1.5 - github.com/gorilla/handlers@v1.4.2 - github.com/gorilla/mux@v1.8.0 - github.com/gorilla/websocket@v1.4.2 - github.com/grpc-ecosystem/grpc-gateway@v1.16.0 - github.com/gsterjov/go-libsecret@v0.0.0-20161001094733-a6f4afe4910c - github.com/gtank/merlin@v0.1.1 - github.com/gtank/ristretto255@v0.1.2 - github.com/hashicorp/golang-lru@v0.5.5-0.20210104140557-80c98217689d - github.com/hashicorp/hcl@v1.0.0 - github.com/holiman/bloomfilter/v2@v2.0.3 - github.com/holiman/uint256@v1.2.0 - github.com/huin/goupnp@v1.0.2 - github.com/jackpal/go-nat-pmp@v1.0.2-0.20160603034137-1fa385a6f458 - github.com/jinzhu/inflection@v1.0.0 - github.com/jinzhu/now@v1.1.4 - github.com/jmespath/go-jmespath@v0.4.0 - github.com/json-iterator/go@v1.1.12 - github.com/lestrrat/go-file-rotatelogs@v0.0.0-20180223000712-d3151e2a480f - github.com/lestrrat/go-strftime@v0.0.0-20180220042222-ba3bf9c1d042 - github.com/libp2p/go-buffer-pool@v0.0.2 - github.com/magiconair/properties@v1.8.1 - github.com/mattn/go-isatty@v0.0.12 - github.com/mattn/go-runewidth@v0.0.9 - github.com/matttproud/golang_protobuf_extensions@v1.0.1 - github.com/mimoo/StrobeGo@v0.0.0-20181016162300-f8f6d4d2b643 - github.com/minio/highwayhash@v1.0.0 - github.com/mitchellh/go-homedir@v1.1.0 - github.com/mitchellh/mapstructure@v1.1.2 - github.com/modern-go/concurrent@v0.0.0-20180306012644-bacd9c7ef1dd - github.com/modern-go/reflect2@v1.0.2 - github.com/mosn/holmes@v0.0.0-20210830110104-685dc05437bf - github.com/mtibben/percent@v0.2.1 - github.com/nacos-group/nacos-sdk-go@v1.0.0 - github.com/olekukonko/tablewriter@v0.0.5 - github.com/orcaman/concurrent-map@v1.0.0 - github.com/pelletier/go-toml@v1.6.0 - github.com/pkg/errors@v0.9.1 - github.com/pmezard/go-difflib@v1.0.0 - github.com/pquerna/ffjson@v0.0.0-20190930134022-aa0246cd15f7 - github.com/prometheus/client_golang@v1.5.1 - github.com/prometheus/client_model@v0.2.0 - github.com/prometheus/common@v0.9.1 - github.com/prometheus/procfs@v0.0.8 - github.com/prometheus/tsdb@v0.7.1 - github.com/rakyll/statik@v0.1.6 - github.com/rcrowley/go-metrics@v0.0.0-20200313005456-10cdbea86bc0 - github.com/regen-network/cosmos-proto@v0.3.1 - github.com/rjeczalik/notify@v0.9.1 - github.com/rs/cors@v1.7.0 - github.com/segmentio/kafka-go@v0.2.2 - github.com/shirou/gopsutil@v3.21.4-0.20210419000835-c7a38de76ee5+incompatible - github.com/spf13/afero@v1.2.1 - github.com/spf13/cast@v1.3.0 - github.com/spf13/cobra@v1.1.1 - github.com/spf13/jwalterweatherman@v1.1.0 - github.com/spf13/pflag@v1.0.5 - github.com/spf13/viper@v1.7.1 - github.com/status-im/keycard-go@v0.0.0-20190424133014-d95853db0f48 - github.com/stretchr/testify@v1.8.0 - github.com/subosito/gotenv@v1.2.0 - github.com/syndtr/goleveldb@v1.0.1-0.20210305035536-64b5b1c73954 - github.com/tendermint/btcd@v0.1.1 - github.com/tendermint/crypto@v0.0.0-20191022145703-50d29ede1e15 - github.com/tendermint/go-amino@v0.15.1 => github.com/okex/go-amino@v0.15.1-okc4 - github.com/tendermint/tm-db@v0.6.7 - github.com/tklauser/go-sysconf@v0.3.9 - github.com/tklauser/numcpus@v0.3.0 - github.com/toolkits/concurrent@v0.0.0-20150624120057-a4371d70e3e3 - github.com/tyler-smith/go-bip39@v1.0.1-0.20181017060643-dbb3b84ba2ef - github.com/valyala/fastjson@v1.6.3 - github.com/willf/bitset@v1.1.11 - go.uber.org/atomic@v1.6.0 - go.uber.org/multierr@v1.5.0 - go.uber.org/zap@v1.15.0 - golang.org/x/crypto@v0.0.0-20210322153248-0c34fe9e7dc2 - golang.org/x/net@v0.0.0-20210805182204-aaa1db679c0d - golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c - golang.org/x/sys@v0.0.0-20220319134239-a9b59b0215f8 - golang.org/x/term@v0.0.0-20201126162022-7de9c90e9dd1 - golang.org/x/text@v0.3.6 - golang.org/x/time@v0.0.0-20210220033141-f8bda1e9f3ba - google.golang.org/genproto@v0.0.0-20200526211855-cb27e3aa2013 - google.golang.org/grpc@v1.44.0 - google.golang.org/protobuf@v1.26.0 - gopkg.in/ini.v1@v1.51.0 - gopkg.in/olebedev/go-duktape.v3@v3.0.0-20200619000410-60c24ae608a6 - gopkg.in/yaml.v2@v2.4.0 - gopkg.in/yaml.v3@v3.0.1 - gorm.io/driver/mysql@v1.3.3 - gorm.io/gorm@v1.23.3 cosmos_sdk: v0.39.2 tendermint: v0.33.9 ```

4. 日志

I[2022-11-01|01:28:32.091][14678] startInProcess. module=main ConsensusStateChainID=exchain-65 GenesisDocChainID=okexchain-65
E[2022-11-01|01:28:32.207][14678] dialing failed (attempts: 1): auth failure: conn.ID (db088cb0b7e7fba0b31e121092bedcc5cd5f7ecc) dialed ID (ad27327fdcf0e9360c61edd4d854513da7838ff7) mismatch. module=pex addr=ad27327fdcf0e9360c61edd4d854513da7838ff7@18.167.77.79:26656
E[2022-11-01|01:28:32.587][14678] dialing failed (attempts: 1): incompatible: peer is on a different network. Got exchain-66, expected okexchain-65. module=pex addr=01b21d39f250a3a5411113aae4a7032eaf9b344e@3.64.37.17:26656
E[2022-11-01|01:28:33.016][14678] Stopping peer for error. module=p2p peer="Peer{MConn{3.13.150.20:36656} b7c6bdfe0c3a6c1c68d6d6849f1b60f566e189dd out}" err=EOF
E[2022-11-01|01:28:33.119][14678] dialing failed (attempts: 1): dial tcp 18.166.245.222:26656: i/o timeout. module=pex addr=7ce43d169e955309e1cca22468ee3ed9e6fd6f45@18.166.245.222:26656
E[2022-11-01|01:28:33.119][14678] dialing failed (attempts: 1): dial tcp 13.231.209.76:26656: i/o timeout. module=pex addr=8842ef204cc20eb5bbb2889a8b70d7f89bf6487b@13.231.209.76:26656
E[2022-11-01|01:28:33.122][14678] Stopping peer for error. module=p2p peer="Peer{MConn{52.221.47.44:36656} 4617e709ddfc6784c5c73ffc85813912df279f9e out}" err=EOF
E[2022-11-01|01:28:33.178][14678] dialing failed (attempts: 1): dial tcp 44.237.38.31:36656: connect: connection refused. module=pex addr=d5f855902f5ab51d353908f7188d0a2175ae73cb@44.237.38.31:36656
E[2022-11-01|01:28:33.181][14678] dialing failed (attempts: 1): duplicate CONN<35.162.198.68:26656>. module=pex addr=0de948586fb30293d1dd14a99ebc3f719deb7c6f@35.162.198.68:26656
E[2022-11-01|01:28:33.184][14678] dialing failed (attempts: 1): duplicate CONN<44.237.38.31:26656>. module=pex addr=96d9cb96041c053e63ff7d0c7d81dfab706136e4@44.237.38.31:26656
E[2022-11-01|01:28:33.343][14678] dialing failed (attempts: 1): auth failure: conn.ID (0de948586fb30293d1dd14a99ebc3f719deb7c6f) dialed ID (daf34fd9fdb611b707a6823ea8fe751f9cd93d31) mismatch. module=pex addr=daf34fd9fdb611b707a6823ea8fe751f9cd93d31@35.162.198.68:26656
E[2022-11-01|01:28:33.345][14678] dialing failed (attempts: 1): auth failure: conn.ID (96d9cb96041c053e63ff7d0c7d81dfab706136e4) dialed ID (f435e5f81db849524850f5ac62c1950bdff01b73) mismatch. module=pex addr=f435e5f81db849524850f5ac62c1950bdff01b73@44.237.38.31:26656
E[2022-11-01|01:28:33.399][14678] Stopping peer for error. module=p2p peer="Peer{MConn{18.177.213.28:36656} 7e64d54f5fef351f74ef3674016d43795f96dc29 out}" err=EOF
I[2022-11-01|01:28:33.645][14678] Height<15757658>, Tx<0>, BlockSize<2108>, RunTx<0ms>, . module=main
panic: Failed to process committed block (15757658:B5AD55C6A29388DE836C8A94FA793818DDF46ED307C96DFF1717A1A95BC1C02D): wrong Block.Header.AppHash.  Expected ABE2F56890E8ABAE8EC4993F15A1403ED8DF804194AEB4FF29D7303DC1250147, got 1582F39D94CE467FD3936B2A4AD6D28F958F926EBD0F3C22610AC8E250C7A405

goroutine 123 [running]:
github.com/okex/exchain/libs/tendermint/blockchain/v0.(*BlockchainReactor).poolRoutine(0xc0001f4000)
        /home/ec2-user/exchain/libs/tendermint/blockchain/v0/reactor.go:394 +0x10ce
created by github.com/okex/exchain/libs/tendermint/blockchain/v0.(*BlockchainReactor).OnStart
        /home/ec2-user/exchain/libs/tendermint/blockchain/v0/reactor.go:133 +0x7a

5. 网络

lisuxiaoqi commented 2 years ago

Thanks for the report. Please use the v1.6.5.1 release and the following snapshot:

ghost commented 2 years ago

The issue is solved by using v1.6.5.1 release and https://okg-pub-hk.oss-cn-hongkong.aliyuncs.com/cdn/oec/snapshot/testnet-s0-fss-20221107-16045201-rocksdb.tar.gz. Thanks for your work!