search

oursky / likecoin-chain

LikeCoin chain node
https://likecoin.bigdipper.live
GNU General Public License v3.0
1 stars 2 forks source link

Statesync fails on v3.0.0 #334

Closed rickmak closed 2 years ago

rickmak commented 2 years ago

I try to setup a new node using state sync and it give following error.

Jul 13 21:21:45 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF VerifyHeader hash=56B4052DAB2FC1967BD9499C71559C5C748E5EB3DCA56E8B30F61CB59B97B046 height=1586001 module=light
Jul 13 21:21:46 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF VerifyHeader hash=FB4E9C87EDAA97B5865E6CCDC6F7B3A3016CB4991FB3161E56EB4B94BB0252D8 height=1586002 module=light
Jul 13 21:21:47 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF Offering snapshot to ABCI app format=1 hash="��V��e\r��Q�����~7p)%�X�~\u007f���P��9" height=1586000 module=statesync
Jul 13 21:21:47 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF Snapshot accepted, restoring format=1 hash="��V��e\r��Q�����~7p)%�X�~\u007f���P��9" height=1586000 module=statesync
Jul 13 21:21:47 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF Fetching snapshot chunk chunk=0 format=1 height=1586000 module=statesync total=1
Jul 13 21:21:47 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF VerifyHeader hash=30EEBA441EB4950F900D881B9F7BE4359F93EFE6E03D8B5A574B58C883AF6870 height=1586000 module=light
Jul 13 21:21:48 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF Header has already been verified hash=56B4052DAB2FC1967BD9499C71559C5C748E5EB3DCA56E8B30F61CB59B97B046 height=1586001 module=light
Jul 13 21:21:48 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF Header has already been verified hash=FB4E9C87EDAA97B5865E6CCDC6F7B3A3016CB4991FB3161E56EB4B94BB0252D8 height=1586002 module=light
Jul 13 21:21:49 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF Header has already been verified hash=56B4052DAB2FC1967BD9499C71559C5C748E5EB3DCA56E8B30F61CB59B97B046 height=1586001 module=light
Jul 13 21:21:49 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF Header has already been verified hash=30EEBA441EB4950F900D881B9F7BE4359F93EFE6E03D8B5A574B58C883AF6870 height=1586000 module=light
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF Applied snapshot chunk to ABCI app chunk=0 format=1 height=1586000 module=statesync total=1
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF Verified ABCI app appHash="�\x03kA�r�A�n�c�q�~� 1��3���T`�Z���" height=1586000 module=statesync
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF Snapshot restored format=1 hash="��V��e\r��Q�����~7p)%�X�~\u007f���P��9" height=1586000 module=statesync
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: 9:21PM INF Starting BlockPool service impl=BlockPool module=blockchain
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: panic: runtime error: invalid memory address or nil pointer dereference
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x141606d]
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: goroutine 255 [running]:
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: github.com/cosmos/cosmos-sdk/x/upgrade.BeginBlocker({{0x7fffe347bd5d, 0x1d}, 0xc001719290, {0x25c0550, 0xc0010af4e0}, {0x25e0718, 0xc000e9f820}, 0xc000e88ea0, {0x25babe0, 0xc000603860}, ...}, ...)
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]:         /go/pkg/mod/github.com/likecoin/cosmos-sdk@v0.45.6-dual-prefix/x/upgrade/abci.go:38 +0x100d
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: github.com/cosmos/cosmos-sdk/x/upgrade.AppModule.BeginBlock(...)
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]:         /go/pkg/mod/github.com/likecoin/cosmos-sdk@v0.45.6-dual-prefix/x/upgrade/module.go:130
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: github.com/cosmos/cosmos-sdk/types/module.(*Manager).BeginBlock(_, {{0x25d5468, 0xc00013c000}, {0x25e1ce0, 0xc003ca4c80}, {{0xb, 0x0}, {0xc00428a500, 0x19}, 0x183351, ...}, ...}, ...)
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]:         /go/pkg/mod/github.com/likecoin/cosmos-sdk@v0.45.6-dual-prefix/types/module/module.go:479 +0x3a2
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: github.com/likecoin/likecoin-chain/v3/app.(*LikeApp).BeginBlocker(_, {{0x25d5468, 0xc00013c000}, {0x25e1ce0, 0xc003ca4c80}, {{0xb, 0x0}, {0xc00428a500, 0x19}, 0x183351, ...}, ...}, ...)
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]:         /go/src/likecoin-chain/app/app.go:588 +0x85
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).BeginBlock(_, {{0xc0052093c0, 0x20, 0x20}, {{0xb, 0x0}, {0xc00428a500, 0x19}, 0x183351, {0x25f44111, ...}, ...}, ...})
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]:         /go/pkg/mod/github.com/likecoin/cosmos-sdk@v0.45.6-dual-prefix/baseapp/abci.go:194 +0x97c
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: github.com/tendermint/tendermint/abci/client.(*localClient).BeginBlockSync(_, {{0xc0052093c0, 0x20, 0x20}, {{0xb, 0x0}, {0xc00428a500, 0x19}, 0x183351, {0x25f44111, ...}, ...}, ...})
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]:         /go/pkg/mod/github.com/tendermint/tendermint@v0.34.19/abci/client/local_client.go:280 +0x118
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: github.com/tendermint/tendermint/proxy.(*appConnConsensus).BeginBlockSync(_, {{0xc0052093c0, 0x20, 0x20}, {{0xb, 0x0}, {0xc00428a500, 0x19}, 0x183351, {0x25f44111, ...}, ...}, ...})
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]:         /go/pkg/mod/github.com/tendermint/tendermint@v0.34.19/proxy/app_conn.go:81 +0x55
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: github.com/tendermint/tendermint/state.execBlockOnProxyApp({0x25d6230?, 0xc00102bbc0}, {0x25db6c0, 0xc000517420}, 0xc0074a21e0, {0x25e0fd8, 0xc0005166d0}, 0x183350?)
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]:         /go/pkg/mod/github.com/tendermint/tendermint@v0.34.19/state/execution.go:307 +0x3dd
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: github.com/tendermint/tendermint/state.(*BlockExecutor).ApplyBlock(_, {{{0xb, 0x0}, {0x1ca9cd8, 0x7}}, {0xc0005384c0, 0x19}, 0x1, 0x183350, {{0xc001612b80, ...}, ...}, ...}, ...)
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]:         /go/pkg/mod/github.com/tendermint/tendermint@v0.34.19/state/execution.go:140 +0x171
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: github.com/tendermint/tendermint/blockchain/v0.(*BlockchainReactor).poolRoutine(0xc0010cc8c0, 0x1)
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]:         /go/pkg/mod/github.com/tendermint/tendermint@v0.34.19/blockchain/v0/reactor.go:398 +0xb5a
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]: created by github.com/tendermint/tendermint/blockchain/v0.(*BlockchainReactor).SwitchToFastSync
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20914]:         /go/pkg/mod/github.com/tendermint/tendermint@v0.34.19/blockchain/v0/reactor.go:125 +0xe5
Jul 13 21:21:52 laichikok.oursky.com cosmovisor[20892]: 9:21PM ERR  error="exit status 2" module=cosmovisor
rickmak commented 2 years ago

I replaced the binary from 3.0.0 to 3.0.0-rc1-hotfix, it works. I though statesync did not run migration...?

I found even after the state is sync, I can't replace 3.0.0-rc1-hotfix with 3.0.0.

elise-ng commented 2 years ago

the error occurred at https://github.com/cosmos/cosmos-sdk/blob/f2d94445c0f5f52cf5ed999b81048b575de94964/x/upgrade/abci.go#L38

which is part of logic to check that the current binary contains the upgrade handler of the last applied upgrade plan recorded in state

so it is expected that the two binaries (v3.0.0-rc1 and the hot fix) are not compatible

rickmak commented 2 years ago

Seem it will only reify on next upgrade.