search

icon-project / btp

32 stars 26 forks source link

Error during deploying which causes Moonbeam exited #95

Open tiendq opened 3 years ago

tiendq commented 3 years ago

An error during deploying, it makes g2m_btp_moonbeam stops. But they were working fine when I restarted all containers after that.

Deployment guide used: https://github.com/icon-project/btp/tree/icondao/docker-compose/goloop2moonbeam

g2m_btp_moonbeam | D|04:12:54.536999|b6b5|-|btp|btp.go:235 canRelay rms:2 has_wait:true skippable:true relayable:false
g2m_btp_moonbeam | P|04:12:56.915627|b6b5|-|btp|btp.go:447 fail to GetResult GetResultParam:&{0x58f5a99b7fb079ac16c36c68c04d4ac0091d1566ad4021d05832e7699829c7c7} err:fail to getresult by pending
g2m_btp_moonbeam | panic: (*logrus.Entry) 0xc0001a5dc0
g2m_btp_moonbeam |
g2m_btp_moonbeam | goroutine 158 [running]:
g2m_btp_moonbeam | github.com/sirupsen/logrus.Entry.log(0xc0001a5f80, 0xc000121140, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
g2m_btp_moonbeam |  /home/ubuntu/work/pkg/mod/github.com/sirupsen/logrus@v1.6.0/entry.go:259 +0x345
g2m_btp_moonbeam | github.com/sirupsen/logrus.(*Entry).Log(0xc000196070, 0x0, 0xc004ae3e70, 0x1, 0x1)
g2m_btp_moonbeam |  /home/ubuntu/work/pkg/mod/github.com/sirupsen/logrus@v1.6.0/entry.go:287 +0xf0
g2m_btp_moonbeam | github.com/sirupsen/logrus.(*Entry).Logf(0xc000196070, 0xc000000000, 0xf9f12d, 0x2b, 0xc000aec6e0, 0x2, 0x2)
g2m_btp_moonbeam |  /home/ubuntu/work/pkg/mod/github.com/sirupsen/logrus@v1.6.0/entry.go:333 +0xe5
g2m_btp_moonbeam | github.com/sirupsen/logrus.(*Entry).Panicf(0xc000196070, 0xf9f12d, 0x2b, 0xc000aec6e0, 0x2, 0x2)
g2m_btp_moonbeam |  /home/ubuntu/work/pkg/mod/github.com/sirupsen/logrus@v1.6.0/entry.go:371 +0x65
g2m_btp_moonbeam | github.com/icon-project/btp/btp.(*BTP).updateResult.func1()
g2m_btp_moonbeam |  /home/ubuntu/testnet/btp/btp/btp.go:447 +0x752
g2m_btp_moonbeam | github.com/gammazero/workerpool.startWorker(0xc005576930, 0xc0009e61e0)
g2m_btp_moonbeam |  /home/ubuntu/work/pkg/mod/github.com/gammazero/workerpool@v1.1.2/workerpool.go:233 +0x27
g2m_btp_moonbeam | created by github.com/gammazero/workerpool.(*WorkerPool).dispatch
g2m_btp_moonbeam |  /home/ubuntu/work/pkg/mod/github.com/gammazero/workerpool@v1.1.2/workerpool.go:195 +0x2f8
g2m_btp_icon    | D|04:12:57.028309|3Cd0|-|btp|btp.go:188 OnBlockOfDst height:235
g2m_btp_moonbeam exited with code 2
trmaphi commented 3 years ago

I found the root cause by ICON node stop way before the BMR shutdown

D|20210914-04:07:55.014415|b6b5|58eb1c|SV|transition.go:422 reportExecution(err=<nil>)
I|20210914-04:07:55.015179|b6b5|58eb1c|SV|transition.go:685 finalizeResult() total=1.525µs world=990ns receipts=535ns
D|20210914-04:07:55.015203|b6b5|58eb1c|SV|transactionmanager.go:86 TM.NotifyFinalized waiters_before=0 waiters_after=0
D|20210914-04:07:55.015330|b6b5|58eb1c|BM|manager.go:1015 Finalize(a1b584115a2ebd731f657440aa8a4fc2ba2f15f330a1e3cdff244d6aec163646)
I|20210914-04:07:55.015818|b6b5|58eb1c|CS|consensus.go:259 enter round Height:407 Round:0
D|20210914-04:07:55.015847|b6b5|58eb1c|CS|consensus.go:305 enterStep {Height:407 Round:0 Step:stepNewHeight}
D|20210914-04:07:56.004615|b6b5|58eb1c|CS|consensus.go:305 enterStep {Height:407 Round:0 Step:stepTransactionWait}
D|20210914-04:07:56.004711|b6b5|58eb1c|CS|consensus.go:305 enterStep {Height:407 Round:0 Step:stepPropose}
D|20210914-04:07:56.004788|b6b5|58eb1c|BM|manager.go:941 Propose(<a1b584115a2ebd731f657440aa8a4fc2ba2f15f330a1e3cdff244d6aec163646>, VoteList(R=0,ID={Count:1,Hash:8c8cda71..},len(Signs)=1))
D|20210914-04:07:56.005158|b6b5|58eb1c|SV|manager.go:248 PatchTransition(patchTxs=<>)
D|20210914-04:07:56.005631|b6b5|58eb1c|SV|transition.go:397 reportValidation(err=<nil>)
D|20210914-04:07:56.005815|b6b5|58eb1c|SV|transition.go:530 Transition.doExecute: height=407 csi=ConsensusInfo(proposer=hxb6b5791be0b5ef67063b3c10b840fb81514db2fd,voters=validatorList[[Validator[addr=hxb6b5791be0b5ef67063b3c10b840fb81514db2fd,pkey=<>]]],voted=[true])
I|20210914-04:07:56.005846|b6b5|58eb1c|SV|transition.go:585 Transactions:      0  Elapsed:     0.044 ms  PerTx:    +Inf µs  TPS:      0.00
D|20210914-04:07:56.005868|b6b5|58eb1c|SV|transition.go:422 reportExecution(err=<nil>)
D|20210914-04:07:56.007859|b6b5|58eb1c|CS|consensus.go:1010 sendProposal ProposalMessage{H:407 R:0 BPSID:{Count:1,Hash:8b49c91c..} Addr:b6b5791b..}
W|20210914-04:07:56.007891|b6b5|58eb1c|CS|consensus.go:1013 sendProposal: NotAvailable
D|20210914-04:07:56.007905|b6b5|58eb1c|BM|manager.go:455 Cancel Propose: Ignored
D|20210914-04:07:56.007923|b6b5|58eb1c|CS|consensus.go:305 enterStep {Height:407 Round:0 Step:stepPrevote}
D|20210914-04:07:56.009698|b6b5|58eb1c|CS|consensus.go:1106 sendVote VoteMessage{PreVote,H:407,R:0,BlockID:5f0b31f7..,Addr:b6b5791b..}
W|20210914-04:07:56.009718|b6b5|58eb1c|CS|consensus.go:1113 sendVote: NotAvailable
D|20210914-04:07:56.009770|b6b5|58eb1c|CS|consensus.go:305 enterStep {Height:407 Round:0 Step:stepPrevoteWait}
D|20210914-04:07:56.009803|b6b5|58eb1c|CS|consensus.go:305 enterStep {Height:407 Round:0 Step:stepPrecommit}
D|20210914-04:07:56.013291|b6b5|58eb1c|CS|consensus.go:1106 sendVote VoteMessage{PreCommit,H:407,R:0,BlockID:5f0b31f7..,Addr:b6b5791b..}
W|20210914-04:07:56.013315|b6b5|58eb1c|CS|consensus.go:1113 sendVote: NotAvailable
D|20210914-04:07:56.013366|b6b5|58eb1c|CS|consensus.go:305 enterStep {Height:407 Round:0 Step:stepPrecommitWait}
D|20210914-04:07:56.013396|b6b5|58eb1c|CS|consensus.go:305 enterStep {Height:407 Round:0 Step:stepCommit}
I|20210914-04:07:56.015522|b6b5|58eb1c|SV|transition.go:685 finalizeResult() total=160.309µs world=155.193µs receipts=5.116µs
D|20210914-04:07:56.015553|b6b5|58eb1c|SV|transactionmanager.go:86 TM.NotifyFinalized waiters_before=0 waiters_after=0
D|20210914-04:07:56.015648|b6b5|58eb1c|BM|manager.go:1015 Finalize(5f0b31f704a49c786831ee4ede25c6584022c8d3ffddb197825c5b8644df1b27)
I|20210914-04:07:56.016544|b6b5|58eb1c|CS|consensus.go:259 enter round Height:408 Round:0
D|20210914-04:07:56.016575|b6b5|58eb1c|CS|consensus.go:305 enterStep {Height:408 Round:0 Step:stepNewHeight}
D|20210914-04:07:57.004821|b6b5|58eb1c|CS|consensus.go:305 enterStep {Height:408 Round:0 Step:stepTransactionWait}
D|20210914-04:07:57.004896|b6b5|58eb1c|CS|consensus.go:305 enterStep {Height:408 Round:0 Step:stepPropose}
D|20210914-04:07:57.004962|b6b5|58eb1c|BM|manager.go:941 Propose(<5f0b31f704a49c786831ee4ede25c6584022c8d3ffddb197825c5b8644df1b27>, VoteList(R=0,ID={Count:1,Hash:8b49c91c..},len(Signs)=1))
D|20210914-04:07:57.005567|b6b5|58eb1c|SV|manager.go:248 PatchTransition(patchTxs=<>)
D|20210914-04:07:57.005763|b6b5|58eb1c|SV|transition.go:397 reportValidation(err=<nil>)
D|20210914-04:07:57.005923|b6b5|58eb1c|SV|transition.go:530 Transition.doExecute: height=408 csi=ConsensusInfo(proposer=hxb6b5791be0b5ef67063b3c10b840fb81514db2fd,voters=validatorList[[Validator[addr=hxb6b5791be0b5ef67063b3c10b840fb81514db2fd,pkey=<>]]],voted=[true])
I|20210914-04:32:42.501666|b6b5|-|cli|server.go:264   ____  ___  _     ___   ___  ____
I|20210914-04:32:42.501867|b6b5|-|cli|server.go:264  / ___|/ _ \| |   / _ \ / _ \|  _ \
I|20210914-04:32:42.501888|b6b5|-|cli|server.go:264 | |  _| | | | |  | | | | | | | |_) |
I|20210914-04:32:42.501905|b6b5|-|cli|server.go:264 | |_| | |_| | |__| |_| | |_| |  __/
I|20210914-04:32:42.501921|b6b5|-|cli|server.go:264  \____|\___/|_____\___/ \___/|_|
I|20210914-04:32:42.501961|b6b5|-|cli|server.go:266 Version : v0.9.9
I|20210914-04:32:42.501987|b6b5|-|cli|server.go:267 Build   : linux/amd64 tags()-2021-07-15-08:03:50
I|20210914-04:32:42.502007|b6b5|-|metric|metric.go:150 Initialize rootMetricCtx
I|20210914-04:32:42.502154|b6b5|-|node|node.go:887 NodeDir : /goloop/data
I|20210914-04:32:42.502176|b6b5|-|node|config.go:116 load  /goloop/data/rconfig.json
I|20210914-04:32:42.502700|b6b5|-|eeproxy|enginefactory.go:9 Allocate Engines:[python java]
D|20210914-04:32:42.502974|b6b5|-|javaee|javaee.go:84 JavaEE Init net(unix), addr(/goloop/data/ee.sock)
D|20210914-04:32:42.503343|b6b5|-|javaee|javaee.go:66 start JavaEE addr(/goloop/data/ee.sock), PID(24), state(0x0),
I|20210914-04:32:42.503439|b6b5|-|pyee|pyee.go:172 start instance uid=0a1885f7-017a-40f6-8aee-072a544fa2c9
I|20210914-04:32:42.503744|b6b5|-|node|node.go:63 Load channel config  /goloop/data/58eb1c/config.json
I|20210914-04:32:42.504311|b6b5|-|chain|chain.go:409 ConfigFilepath /goloop/data/58eb1c/config.json BaseDir  ChainDir /goloop/data/58eb1c
I|20210914-04:32:42.504355|b6b5|58eb1c|chain|chain.go:338 prepare a directory /goloop/data/58eb1c/db for database