search

cosmos / gaia

Cosmos Hub
https://hub.cosmos.network
Apache License 2.0
466 stars 681 forks source link

Gaia 9.0.1: segfault #2289

Closed fmira21 closed 1 year ago

fmira21 commented 1 year ago

Hi guys! Just updated my Dockerised node to 9.0.1 and facing this issue:

12:02PM INF starting node with ABCI Tendermint in-process
12:02PM INF service start impl=multiAppConn module=proxy msg={}
12:02PM INF service start connection=query impl=localClient module=abci-client msg={}
12:02PM INF service start connection=snapshot impl=localClient module=abci-client msg={}
12:02PM INF service start connection=mempool impl=localClient module=abci-client msg={}
12:02PM INF service start connection=consensus impl=localClient module=abci-client msg={}
12:02PM INF service start impl=EventBus module=events msg={}
12:02PM INF service start impl=PubSub module=pubsub msg={}
12:04PM INF service start impl=IndexerService module=txindex msg={}
12:04PM INF Found local state with non-zero height, skipping state sync
12:04PM INF ABCI Handshake App Info hash="��,T����n\"\x1eB��Q\x03j�\v0\"����YN\t\r\x1f��" height=14459094 module=consensus protocol-version=0 software-version=v9.0.1
12:04PM INF ABCI Replay Blocks appHeight=14459094 module=consensus stateHeight=14459094 storeHeight=14459094
12:04PM INF Completed ABCI Handshake - Tendermint and App are synced appHash="��,T����n\"\x1eB��Q\x03j�\v0\"����YN\t\r\x1f��" appHeight=14459094 module=consensus
12:04PM INF Version info block=11 p2p=8 tendermint_version=v0.34.26
12:04PM INF This node is not a validator addr=6AC3862F09D7D930FA4C75D85F776ECDE679FF8F module=consensus pubKey=oiEy7gCv7v+Ub2yd0slO7a9CFW2eKdl9TCoSDuO45Ts=
12:04PM INF P2P Node ID ID=045303fa185c9c33ea02a5a2807507bc3451b4fc file=/root/.gaia/node_key.json module=p2p
12:04PM INF Adding persistent peers addrs=["6e08b23315a9f0e1b23c7ed847934f7d6f848c8b@165.232.156.86:26656","ee27245d88c632a556cf72cc7f3587380c09b469@45.79.249.253:26656","538ebe0086f0f5e9ca922dae0462cc87e22f0a50@34.122.34.67:26656","d3209b9f88eec64f10555a11ecbf797bb0fa29f4@34.125.169.233:26656","bdc2c3d410ca7731411b7e46a252012323fbbf37@34.83.209.166:26656","585794737e6b318957088e645e17c0669f3b11fc@54.160.123.34:26656","5b4ed476e01c49b23851258d867cc0cfc0c10e58@206.189.4.227:26656"] module=p2p
12:04PM INF Adding unconditional peer ids ids=[] module=p2p
12:04PM INF Add our address to book addr={"id":"045303fa185c9c33ea02a5a2807507bc3451b4fc","ip":"35.178.23.46","port":26656} book=/root/.gaia/config/addrbook.json module=p2p
12:04PM INF Add our address to book addr={"id":"045303fa185c9c33ea02a5a2807507bc3451b4fc","ip":"0.0.0.0","port":26656} book=/root/.gaia/config/addrbook.json module=p2p
12:04PM INF service start impl=Node msg={}
12:04PM INF service start impl="P2P Switch" module=p2p msg={}
12:04PM INF service start impl=BlockchainReactor module=blockchain msg={}
12:04PM INF service start impl=BlockPool module=blockchain msg={}
12:04PM INF service start impl=ConsensusReactor module=consensus msg={}
12:04PM INF Reactor  module=consensus waitSync=true
12:04PM INF service start impl=Evidence module=evidence msg={}
12:04PM INF service start impl=StateSync module=statesync msg={}
12:04PM INF service start impl=PEX module=pex msg={}
12:04PM INF service start book=/root/.gaia/config/addrbook.json impl=AddrBook module=p2p msg={}
12:04PM INF serve module=rpc-server msg={}
12:04PM ERR Connecting to seed failed err="error looking up host (cosmos-cosmoshub-3-seed.nodes.polychainlabs.com): lookup cosmos-cosmoshub-3-seed.nodes.polychainlabs.com on 127.0.0.11:53: no such host" module=pex
12:04PM INF Saving AddrBook to file book=/root/.gaia/config/addrbook.json module=p2p size=1078
12:04PM INF Ensure peers module=pex numDialing=0 numInPeers=0 numOutPeers=0 numToDial=10
12:04PM INF service start impl="Peer{MConn{74.118.139.150:26656} 44390f449904199d22148d07822b1462048a38ef out}" module=p2p msg={} peer={"id":"44390f449904199d22148d07822b1462048a38ef","ip":"74.118.139.150","port":26656}
12:04PM INF service start impl=MConn{74.118.139.150:26656} module=p2p msg={} peer={"id":"44390f449904199d22148d07822b1462048a38ef","ip":"74.118.139.150","port":26656}
12:04PM INF service start impl="Peer{MConn{178.18.249.59:26656} 8d6f78575440532fe7b04fb206f18b3abe1cc9e7 out}" module=p2p msg={} peer={"id":"8d6f78575440532fe7b04fb206f18b3abe1cc9e7","ip":"178.18.249.59","port":26656}
12:04PM INF service start impl=MConn{178.18.249.59:26656} module=p2p msg={} peer={"id":"8d6f78575440532fe7b04fb206f18b3abe1cc9e7","ip":"178.18.249.59","port":26656}
12:04PM INF Connection is closed @ recvRoutine (likely by the other side) conn={"Logger":{}} module=p2p peer={"id":"8d6f78575440532fe7b04fb206f18b3abe1cc9e7","ip":"178.18.249.59","port":26656}
12:04PM INF service stop impl={"Logger":{}} module=p2p msg={} peer={"id":"8d6f78575440532fe7b04fb206f18b3abe1cc9e7","ip":"178.18.249.59","port":26656}
12:04PM ERR Stopping peer for error err=EOF module=p2p peer={"Data":{},"Logger":{}}
12:04PM INF service stop impl={"Data":{},"Logger":{}} module=p2p msg={} peer={"id":"8d6f78575440532fe7b04fb206f18b3abe1cc9e7","ip":"178.18.249.59","port":26656}
12:04PM INF service start impl="Peer{MConn{135.181.188.17:26656} 0393c19b176d1cf8bc560c5a8fa990301deb1a7e out}" module=p2p msg={} peer={"id":"0393c19b176d1cf8bc560c5a8fa990301deb1a7e","ip":"135.181.188.17","port":26656}
12:04PM INF service start impl=MConn{135.181.188.17:26656} module=p2p msg={} peer={"id":"0393c19b176d1cf8bc560c5a8fa990301deb1a7e","ip":"135.181.188.17","port":26656}
12:04PM INF service start impl="Peer{MConn{172.93.214.10:26656} 58b54d8cfdc0c634ed592e2c008705791253ebbb out}" module=p2p msg={} peer={"id":"58b54d8cfdc0c634ed592e2c008705791253ebbb","ip":"172.93.214.10","port":26656}
12:04PM INF service start impl=MConn{172.93.214.10:26656} module=p2p msg={} peer={"id":"58b54d8cfdc0c634ed592e2c008705791253ebbb","ip":"172.93.214.10","port":26656}
12:04PM INF Connection is closed @ recvRoutine (likely by the other side) conn={"Logger":{}} module=p2p peer={"id":"58b54d8cfdc0c634ed592e2c008705791253ebbb","ip":"172.93.214.10","port":26656}
12:04PM INF service stop impl={"Logger":{}} module=p2p msg={} peer={"id":"58b54d8cfdc0c634ed592e2c008705791253ebbb","ip":"172.93.214.10","port":26656}
12:04PM ERR Stopping peer for error err=EOF module=p2p peer={"Data":{},"Logger":{}}
12:04PM INF service stop impl={"Data":{},"Logger":{}} module=p2p msg={} peer={"id":"58b54d8cfdc0c634ed592e2c008705791253ebbb","ip":"172.93.214.10","port":26656}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x159ec4d]
goroutine 92 [running]:
github.com/cosmos/cosmos-sdk/x/upgrade.BeginBlocker({{0xc00114d7d0, 0xb}, 0xc0067ec810, {0x2751338, 0xc006829230}, {0x2771590, 0xc001169cc0}, 0xc0068560f0, {0x274af60, 0xc000654000}, ...}, ...)
    github.com/cosmos/cosmos-sdk@v0.45.13-ics/x/upgrade/abci.go:38 +0x100d
github.com/cosmos/cosmos-sdk/x/upgrade.AppModule.BeginBlock(...)
    github.com/cosmos/cosmos-sdk@v0.45.13-ics/x/upgrade/module.go:130
github.com/cosmos/cosmos-sdk/types/module.(*Manager).BeginBlock(_, {{0x27655a8, 0xc000056030}, {0x2771148, 0xc11706d700}, {{0xb, 0x0}, {0xc113709170, 0xb}, 0xdca0d7, ...}, ...}, ...)
    github.com/cosmos/cosmos-sdk@v0.45.13-ics/types/module/module.go:491 +0x3bb
github.com/cosmos/gaia/v9/app.(*GaiaApp).BeginBlocker(_, {{0x27655a8, 0xc000056030}, {0x2771148, 0xc11706d700}, {{0xb, 0x0}, {0xc113709170, 0xb}, 0xdca0d7, ...}, ...}, ...)
    github.com/cosmos/gaia/v9/app/app.go:247 +0x85
github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).BeginBlock(_, {{0xc010463c20, 0x20, 0x20}, {{0xb, 0x0}, {0xc113709170, 0xb}, 0xdca0d7, {0xae67f53, ...}, ...}, ...})
    github.com/cosmos/cosmos-sdk@v0.45.13-ics/baseapp/abci.go:177 +0x97c
github.com/tendermint/tendermint/abci/client.(*localClient).BeginBlockSync(_, {{0xc010463c20, 0x20, 0x20}, {{0xb, 0x0}, {0xc113709170, 0xb}, 0xdca0d7, {0xae67f53, ...}, ...}, ...})
    github.com/tendermint/tendermint@v0.34.26/abci/client/local_client.go:280 +0x118
github.com/tendermint/tendermint/proxy.(*appConnConsensus).BeginBlockSync(_, {{0xc010463c20, 0x20, 0x20}, {{0xb, 0x0}, {0xc113709170, 0xb}, 0xdca0d7, {0xae67f53, ...}, ...}, ...})
    github.com/tendermint/tendermint@v0.34.26/proxy/app_conn.go:81 +0x55
github.com/tendermint/tendermint/state.execBlockOnProxyApp({0x27663e0?, 0xc00009bbc0}, {0x276be60, 0xc013e43df0}, 0xc115c5d860, {0x2772ad8, 0xc02116c180}, 0xdca0d6?)
    github.com/tendermint/tendermint@v0.34.26/state/execution.go:307 +0x3dd
github.com/tendermint/tendermint/state.(*BlockExecutor).ApplyBlock(_, {{{0xb, 0x0}, {0xc101536af0, 0x8}}, {0xc101536b00, 0xb}, 0x4f5b97, 0xdca0d6, {{0xc01ecaed40, ...}, ...}, ...}, ...)
    github.com/tendermint/tendermint@v0.34.26/state/execution.go:140 +0x171
github.com/tendermint/tendermint/blockchain/v0.(*BlockchainReactor).poolRoutine(0xc000655a40, 0x0)
    github.com/tendermint/tendermint@v0.34.26/blockchain/v0/reactor.go:400 +0xbda
created by github.com/tendermint/tendermint/blockchain/v0.(*BlockchainReactor).OnStart
    github.com/tendermint/tendermint@v0.34.26/blockchain/v0/reactor.go:112 +0x7a

Looks like I saw the same segfault on 8.0.1 when it was just released. Is this a bug? When exactly should I apply new updates?

Thank you in advance!

mmulji-ic commented 1 year ago

hi @fmira21 v9.0.x will be live tomorrow, please see the proposal https://www.mintscan.io/cosmos/proposals/187. Until that time you need to use an v8.0.x series.

fmira21 commented 1 year ago

@mmulji-ic Thank you for the answer!