synechron-finlabs / quorum-maker

Utility to create and monitor Quorum nodes
Apache License 2.0
196 stars 97 forks source link

Nodes Data Does Not Sync On Public Transactions After Docker Images Down #76

Closed muhammetevirgen closed 5 years ago

muhammetevirgen commented 5 years ago

I'm running a 4 nodes setup with QM, each on a different Ubuntu machine on Amazon. I am trying some test cases. When ı create a network from scratch with 1 creator and 3 joiner, everything is fine.

But when ı want to test if 1 of the node goes down (removing docker contanier) and trying to rejoin network with same node name and IP address the nodes come back successfully but the data is not synced.

I am runnnig start.sh under nodename folder , node come back but the data is not synced from the creator node or from any other.

Is there a way to rejoin existing network with same IP and same node name with whole data sync ?

dhyansraj commented 5 years ago

Are you joining the same node using 2. Join network option again? Once joined, you are only supposed to stop and start node, not join again. You can change to respective node directory and execute ./start.sh to bring the node up.

When you create the first node, it has already ahead in 2 transactions (Node Manager contract deployment and self register transaction). When you join the second node, it must have synced the chain and caught up the blocks. After joining the second node, you would mostly see 3 blocks on all 2 nodes. So I believe chain syncing is happening correctly (If not you wont be able to see both nodes correctly on the UI).

Please let me know the steps you followed in details so that I can help you further.

muhammetevirgen commented 5 years ago

Hi Again,

What I have done is , I have re run my test setting up 3 nodes correctly, there was no sync problem at begining. Afterward I have deployed my own test contract and which went well on all nodes, all nodes was synced and had 8 blocks in my example. Then I reboot one of the node (which will cause to docker stop I think , I dont see anything on docker ps -a after reboot), and all others lost connection on healtcheck as expected. After reboot, I enter the node directory folder and run start.sh command to start node again. When I check the UI on rebooted node, I see that other 2 nodes are getting heartbeart from new comer and update the node count as 3. But the main problem is,on the UI for rebooted node there is no block.

I am attaching the list below.

1) QM UI screenshot of rebooted node after reboot(Rebooted_Node_UI_Screenshot.png) 2) QM UI screenshot of node1 which is creator node(Creator_Node_UI_Screenshot.png) 3) QM UI screenshot of node2 which is another healthy node on network(Second_Node_UI_Screenshot.png) 4) Geth log of rebooted node (Rebooted_Node_Geth_Log.txt) 5) Geth log creator node (Creator_Node_Geth_Log.log.txt) 6) Geth log of second node (Second_Node_Geth_Log.log.txt)

Creator_Node_Geth_Log.log.txt Rebooted_Node_Geth_Log.txt

creator_node_ui_screenshot rebooted_node_ui_screenshot second_node_ui_screenshot

Second_Node_Geth_Log.log.txt

dhyansraj commented 5 years ago

When you rebooted the node, did the IP change ? Is this hosted on cloud?

muhammetevirgen commented 5 years ago

This host is on cloud but I have static IP address so IP address of host did not change after reboot

dhyansraj commented 5 years ago

This issue is related to #87 and is fixed now. Can you try again?

muhammetevirgen commented 5 years ago

I tried again and looks issue is fixed !

muhammetevirgen commented 5 years ago

Fixed