search

crypto-org-chain / cronos

Cronos is the first Ethereum-compatible blockchain network built on Cosmos SDK technology. Cronos aims to massively scale the DeFi, GameFi, and overall Web3 user community by providing builders with the ability to instantly port apps and crypto assets from other chains while benefiting from low transaction fees, high throughput, and fast finality.
Other
287 stars 228 forks source link

Cronos Mainnet node stopped getting new blocks after 12089664 #1296

Open Aracki opened 5 months ago

Aracki commented 5 months ago

Version: v1.0.11-hardfork

One of two Cronos Mainnet nodes stopped getting new blocks after 12089664. It got:

6:17AM ERR prevote step: ProposalBlock is invalid err="wrong Block.Header.AppHash. Expected 8DF599A79B6799E160AEB9B26438A0B70428A660EFF77D12C4CE731C7A8B142E, got C7AD6CE2DFF0EC2D0558E4CDE1EBF329446A6FFB3C77EB9B90A4409CA7F34272" height=12089664 module=consensus round=0 server=node

6:17AM ERR CONSENSUS FAILURE!!! err="precommit step; +2/3 prevoted for an invalid block: wrong Block.Header.AppHash. Expected 8DF599A79B6799E160AEB9B26438A0B70428A660EFF77D12C4CE731C7A8B142E, got C7AD6CE2DFF0EC2D0558E4CDE1EBF329446A6FFB3C77EB9B90A4409CA7F34272" module=consensus server=node stack="goroutine 140271667 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/tendermint/tendermint/consensus.(State).receiveRoutine.func2()\n\tgithub.com/tendermint/tendermint/consensus/state.go:732 +0x4c\npanic({0x2d1d8a0, 0xc06802d060})\n\truntime/panic.go:884 +0x212\ngithub.com/tendermint/tendermint/consensus.(State).enterPrecommit(0xc001d62000, 0xb87940, 0x0)\n\tgithub.com/tendermint/tendermint/consensus/state.go:1414 +0x1739\ngithub.com/tendermint/tendermint/consensus.(State).addVote(0xc001d62000, 0xc05ccec1e0, {0xc0320e7f50, 0x28})\n\tgithub.com/tendermint/tendermint/consensus/state.go:2137 +0x13d8\ngithub.com/tendermint/tendermint/consensus.(State).tryAddVote(0xc001d62000, 0xc05ccec1e0, {0xc0320e7f50?, 0xc0235fdd00?})\n\tgithub.com/tendermint/tendermint/consensus/state.go:1963 +0x2c\ngithub.com/tendermint/tendermint/consensus.(State).handleMsg(0xc001d62000, {{0x4347a60?, 0xc009288fc8?}, {0xc0320e7f50?, 0x0?}})\n\tgithub.com/tendermint/tendermint/consensus/state.go:861 +0x170\ngithub.com/tendermint/tendermint/consensus.(State).receiveRoutine(0xc001d62000, 0x0)\n\tgithub.com/tendermint/tendermint/consensus/state.go:768 +0x3f9\ncreated by github.com/tendermint/tendermint/consensus.(*State).OnStart\n\tgithub.com/tendermint/tendermint/consensus/state.go:379 +0x12d\n"

yihuang commented 5 months ago

Can't see what happened, can you try cronosd rollback then resync with latest v1.0.x binary

Aracki commented 5 months ago

Already tried. Rollback is running for more than a day with no output at all.