search

quantadex / distributed_quanta_bridge

The distributed version of the quanta bridge
1 stars 0 forks source link

Deposit does not work when a node is stopped and restarted again. (non-leader node) #21

Closed shahpoo415 closed 5 years ago

shahpoo415 commented 5 years ago

Run 3 nodes Submit Transaction Stop one node Submit Transaction Restart the node



2018/11/30 11:44:34 I [5000] Start new round 0x3fd5e39ebf4742e61bd9db018e2c663c01e629a661e15f8043e4a123e2dd7b58 ETH to=QBBWTYBD54J5OX7RO32TBFT75527CZQTX5MTUY267H7K5EC7HRIUNTDV amount=400000
Propose TX:  ETH QDFLBGRMXEDTLI2LBYQTF7ZX4LPGOWIBN2UDQOZW6VH47LNEX5MEXBST 0.0400000 QBBWTYBD54J5OX7RO32TBFT75527CZQTX5MTUY267H7K5EC7HRIUNTDV
2018/11/30 11:44:34 I [5000] Got blocks [8071 8072]
2018/11/30 11:44:34 I [5000] ***** Start # of blocks=2 man.N=3,man.Q=3 *** 
2018/11/30 11:44:34 I [5000] set last block coin=ETH height=8072
2018/11/30 11:44:34 I [5000] QuantaToCoin refunds []
2018/11/30 11:44:34 I [5000] Next cursor is = 4779830449016833, numRefunds=0
2018/11/30 11:44:34 I [cosi] Start new round leader=true nodes=3 threshold=3
2018/11/30 11:44:34 I [cosi] Got commitment 2/3
2018/11/30 11:44:34 I [cosi] Got commitment 3/3
2018/11/30 11:44:34 I [cosi] Got total of 3 commitments, moving forward
2018/11/30 11:44:34 I [5000] Sign msg wGGc8QAAAEBswnde3ksmBxQ1+utUxt3XmtPfAzDGNcrLJMYe2Ydzy0h6U5W2xjdswuNW8EY0XaLR34MUoejeSysazqupKKcL
2018/11/30 11:44:34 I [cosi] Got signature 2/3
2018/11/30 11:44:34 I [cosi] Got signature 3/3
2018/11/30 11:44:34 I [5000] Great! Cosi successfully signed deposit
2018/11/30 11:44:34 E [5000] processSubmissions pending=1
2018/11/30 11:44:34 I [5000] Submit TX: 0x3fd5e39ebf4742e61bd9db018e2c663c01e629a661e15f8043e4a123e2dd7b58 signed=true AAAAAMqwmiy5BzWjSw4hMv834t5nWQFuqDg7NvVPz62kv1hLAAAAZAAPDxkAAAAqAAAAAAAAAAAAAAABAAAAAAAAAAEAAAAAQ2ngI+8T11/xdvUwln/vdfFmE79ZOmNe+f6ukF88UUYAAAABRVRIAAAAAADKsJosuQc1o0sOITL/N+LeZ1kBbqg4Ozb1T8+tpL9YSwAAAAAABhqAAAAAAAAAAAPAYZzxAAAAQGzCd17eSyYHFDX661TG3dea098DMMY1ysskxh7Zh3PLSHpTlbbGN2zC41bwRjRdotHfgxSh6N5LKxrOq6kopwunC6puAAAAQBNZ9o3r+5Vl1ybLT+f2A3/ZlM3gYVy/7n6/pm6oxS6ZMeLHNV7+iWZbd81KUw+9N1rAoww9wM5c80OIYh4lggEWQn5KAAAAQIoErPsnbGmnjGcrK/WGIaBT6hBfdWku2iUJiFsJLmaY5XT8xdfj/BCs1K4Kg22SfVpVLcMBLwSg/TSy0a0KrAA=
2018/11/30 11:44:37 I [5000] Successful tx submission d2f553d2024c13239289fe201f58e36b8c6649e5ab90738d2467b7fd608bcec2,remove %!s(int=0)

Stopped node3(Leader Log)

2018/11/30 11:59:25 I [cosi] Start new round leader=true nodes=3 threshold=3
Error! Post http://0.0.0.0:5002/node/api/peer: dial tcp 0.0.0.0:5002: connect: connection refused
2018/11/30 11:59:25 I [cosi] Got commitment 2/3
2018/11/30 11:59:25 D [5000] Coin2Quanta: No new block last=8136 top=8136
2018/11/30 11:59:25 I [5000] ***** Start # of blocks=0 man.N=3,man.Q=3 *** 
2018/11/30 11:59:25 I [5000] QuantaToCoin refunds []
2018/11/30 11:59:25 I [5000] Next cursor is = 4779830449016833, numRefunds=0
2018/11/30 11:59:28 E [cosi] Timeout should not happen received 2/3
2018/11/30 11:59:28 E [cosi] Bailing out of leader prep round, not enough nodes
2018/11/30 11:59:28 E [5000] Unable to sign for refund
2018/11/30 11:59:28 D [5000] Coin2Quanta: No new block last=8136 top=8136
2018/11/30 11:59:28 I [5000] ***** Start # of blocks=0 man.N=3,man.Q=3 *** 
2018/11/30 11:59:28 I [5000] QuantaToCoin refunds []
2018/11/30 11:59:28 I [5000] Next cursor is = 4779830449016833, numRefunds=0
2018/11/30 11:59:29 I [5000] Start new round 0xc2cde8f9ae68a4ee1f07edb7b347155683709b0f9d469d136c815ae2e5f37571 ETH to=QBBWTYBD54J5OX7RO32TBFT75527CZQTX5MTUY267H7K5EC7HRIUNTDV amount=600000
Propose TX:  ETH QDFLBGRMXEDTLI2LBYQTF7ZX4LPGOWIBN2UDQOZW6VH47LNEX5MEXBST 0.0600000 QBBWTYBD54J5OX7RO32TBFT75527CZQTX5MTUY267H7K5EC7HRIUNTDV

Restarted node3 (Leader Log)

2018/11/30 11:59:45 I [cosi] Start new round leader=true nodes=3 threshold=3
2018/11/30 11:59:45 I [cosi] Got commitment 2/3
2018/11/30 11:59:47 D [5000] Coin2Quanta: No new block last=8138 top=8138
2018/11/30 11:59:47 I [5000] ***** Start # of blocks=0 man.N=3,man.Q=3 *** 
2018/11/30 11:59:47 I [5000] QuantaToCoin refunds []
2018/11/30 11:59:47 I [5000] Next cursor is = 4779830449016833, numRefunds=0
2018/11/30 11:59:48 E [cosi] Timeout should not happen received 2/3
2018/11/30 11:59:48 E [cosi] Bailing out of leader prep round, not enough nodes
2018/11/30 11:59:48 E [5000] Unable to sign for refund
2018/11/30 11:59:49 I [5000] Start new round 0xc2cde8f9ae68a4ee1f07edb7b347155683709b0f9d469d136c815ae2e5f37571 ETH to=QBBWTYBD54J5OX7RO32TBFT75527CZQTX5MTUY267H7K5EC7HRIUNTDV amount=600000
Propose TX:  ETH QDFLBGRMXEDTLI2LBYQTF7ZX4LPGOWIBN2UDQOZW6VH47LNEX5MEXBST 0.0600000 QBBWTYBD54J5OX7RO32TBFT75527CZQTX5MTUY267H7K5EC7HRIUNTDV
2018/11/30 11:59:49 I [cosi] Start new round leader=true nodes=3 threshold=3
2018/11/30 11:59:50 I [5000] Got blocks [8139 8140]
2018/11/30 11:59:50 I [5000] ***** Start # of blocks=2 man.N=3,man.Q=3 *** 
2018/11/30 11:59:50 I [5000] set last block coin=ETH height=8140
2018/11/30 11:59:51 I [5000] QuantaToCoin refunds []
2018/11/30 11:59:51 I [5000] Next cursor is = 4779830449016833, numRefunds=0
2018/11/30 11:59:52 E [cosi] Timeout should not happen received 1/3
2018/11/30 11:59:52 E [cosi] Bailing out of leader prep round, not enough nodes
2018/11/30 11:59:52 E [5000] Unable to sign for refund
2018/11/30 11:59:53 I [5000] Start new round 0xc2cde8f9ae68a4ee1f07edb7b347155683709b0f9d469d136c815ae2e5f37571 ETH to=QBBWTYBD54J5OX7RO32TBFT75527CZQTX5MTUY267H7K5EC7HRIUNTDV amount=600000
Propose TX:  ETH QDFLBGRMXEDTLI2LBYQTF7ZX4LPGOWIBN2UDQOZW6VH47LNEX5MEXBST 0.0600000 QBBWTYBD54J5OX7RO32TBFT75527CZQTX5MTUY267H7K5EC7HRIUNTDV
2018/11/30 11:59:53 I [cosi] Start new round leader=true nodes=3 threshold=3
2018/11/30 11:59:54 D [5000] Coin2Quanta: No new block last=8140 top=8140
2018/11/30 11:59:54 I [5000] ***** Start # of blocks=0 man.N=3,man.Q=3 *** 
2018/11/30 11:59:54 I [5000] QuantaToCoin refunds []
2018/11/30 11:59:54 I [5000] Next cursor is = 4779830449016833, numRefunds=0
2018/11/30 11:59:56 E [cosi] Timeout should not happen received 1/3
2018/11/30 11:59:56 E [cosi] Bailing out of leader prep round, not enough nodes
2018/11/30 11:59:56 E [5000] Unable to sign for refund

Node3 log when restarted

2018/11/30 11:59:39 I [5002] Initialize ledger
2018/11/30 11:59:39 I [5002] REST API started at :0...
2018/11/30 11:59:40 I [5002] Added to quorum
2018/11/30 11:59:40 I [5002] Trust initialized
2018/11/30 11:59:41 I [5002] Got blocks [8135 8136 8137 8138]
2018/11/30 11:59:41 I [5002] ***** Start # of blocks=4 man.N=3,man.Q=3 *** 
2018/11/30 11:59:41 I [5002] Block 8135 Got deposits 1 [0xc0004b48c0]
2018/11/30 11:59:41 E [cosi] Unable to validate transactions: %!(EXTRA string=Unable to verify: 0xc2cde8f9ae68a4ee1f07edb7b347155683709b0f9d469d136c815ae2e5f37571 not found)
2018/11/30 11:59:42 I [5002] set last block coin=ETH height=8138
2018/11/30 11:59:42 I [5002] QuantaToCoin refunds []
2018/11/30 11:59:42 I [5002] Next cursor is = 4779830449016833, numRefunds=0
2018/11/30 11:59:45 D [5002] Coin2Quanta: No new block last=8138 top=8138
2018/11/30 11:59:45 I [5002] ***** Start # of blocks=0 man.N=3,man.Q=3 *** 
2018/11/30 11:59:45 I [5002] QuantaToCoin refunds []
2018/11/30 11:59:45 I [5002] Next cursor is = 4779830449016833, numRefunds=0```
quocble commented 5 years ago

@shahpoo415

It looks like the restarted node wasn't able to verify 0xc2cde8f9ae68a4ee1f07edb7b347155683709b0f9d469d136c815ae2e5f37571

Can you check whether node3 restarted, resumed, and processed the block with # 0xc2cde8f9ae68a4ee1f07edb7b347155683709b0f9d469d136c815ae2e5f37571

shahpoo415 commented 5 years ago

Block 8135 has this transaction. Node3 is processing that block. 2018/11/30 11:59:41 I [5002] Got blocks [8135 8136 8137 8138] 2018/11/30 11:59:41 I [5002] ** Start # of blocks=4 man.N=3,man.Q=3 2018/11/30 11:59:41 I [5002] Block 8135 Got deposits 1 [0xc0004b48c0]

quocble commented 5 years ago

The leader should be trying it again, which I see from the leader log.... Does node 3 receive anymore COSI message?

shahpoo415 commented 5 years ago

It is getting "Follower got prepare 3/3" and the number keeps increasing.

2018/11/30 11:59:49 I [cosi] Follower got prepare 3/3 2018/11/30 11:59:52 D [5002] Coin2Quanta: No new block last=8140 top=8140 2018/11/30 11:59:52 I [5002] ** Start # of blocks=0 man.N=3,man.Q=3 2018/11/30 11:59:52 I [5002] QuantaToCoin refunds [] 2018/11/30 11:59:52 I [5002] Next cursor is = 4779830449016833, numRefunds=0 2018/11/30 11:59:53 I [cosi] Follower got prepare 4/3 2018/11/30 11:59:55 D [5002] Coin2Quanta: No new block last=8140 top=8140 2018/11/30 11:59:55 I [5002] ** Start # of blocks=0 man.N=3,man.Q=3 2018/11/30 11:59:55 I [5002] QuantaToCoin refunds [] 2018/11/30 11:59:55 I [5002] Next cursor is = 4779830449016833, numRefunds=0 2018/11/30 11:59:57 I [cosi] Follower got prepare 5/3

quocble commented 5 years ago

Run

dep ensure -update (in parent directory)

There's a bug fix already I made on quanta_book