synechron-finlabs / quorum-maker

Utility to create and monitor Quorum nodes
Apache License 2.0
196 stars 97 forks source link

Can't connect two Nodes on two machines in AWS #54

Closed YouNNN closed 5 years ago

YouNNN commented 5 years ago

Using centOS 7 on ec2 instances with the latest quorum-maker. Wanted to connect a Node from instance A with a Node from instance B with the "join network" task. I configured the Network-Node and can access the webinterface.

Node A: Please select an option: 1) Create Network 2) Join Network 3) Attach to an existing Node 4) Setup Development/Test Network 5) Exit option: 1 Please enter node name: nodename1 Please enter IP Address of this node: myip Please enter RPC Port of this node[Default:22000]: Please enter Network Listening Port of this node[Default:22001]: Please enter Constellation Port of this node[Default:22002]: Please enter Raft Port of this node[Default:22003]: Please enter Node Manager Port of this node[Default:22004]: Please enter WS Port of this node[Default:22005]:


Successfully created and started nodename1 You can send transactions to myip:22000 For private transactions, use pTRZo0CoFY8z8Og977Il8EJt7NkIQe0zETP6rLEPmW8= For accessing Quorum Maker UI, please open the following from a web browser http://localhost:22004/ To join this node from a different host, please run Quorum Maker and choose option to run Join Network When asked, enter myip for Existing Node IP and 22004 for Node Manager Port


{"level":"info","msg":"Node Manager listening on :22004...","time":"2018-10-02T09:56:19Z"} {"level":"info","msg":"Deploying Network Manager Contract","time":"2018-10-02T09:56:20Z"} {"level":"info","msg":"Adding whitelisted IPs","time":"2018-10-02T09:56:23Z"} ^C{"level":"info","msg":"Node Manager Shutting down","time":"2018-10-02T10:13:19Z"}

Node B: Please select an option: 1) Create Network 2) Join Network 3) Attach to an existing Node 4) Setup Development/Test Network 5) Exit option: 2 Please enter node name: noden12 Please enter IP Address of existing node: myip Please enter Node Manager Port of existing node: 22004
Please enter IP Address of this node: myotherip Please enter RPC Port of this node[Default:22000]:23000 Please enter Network Listening Port of this node[Default:23001]: Please enter Constellation Port of this node[Default:23002]: Please enter Raft Port of this node[Default:23003]: Please enter Node Manager Port of this node[Default:23004]: Please enter WS Port of this node[Default:23005]:

Join Request sent to myip. Waiting for approval...

and then it gets stuck and times out. I don't get a notification for a approval. is this an error with docker and ports on the instance? Edit: There is no file in qdata/logs, so I don't know whats the error.

karthik137 commented 5 years ago

Check your docker container. Do

$docker ps -a

Check container status. If it is running then do $ netstat -pnat | grep listen

Check rpc and nodemanager status.

Can u please post the logs?

YouNNN commented 5 years ago

Hey! :) so the Node, wich is hosting: docker ps -a WARNING: Error loading config file:/home/uib05246/.docker/config.json - invalid character 'G' looking for beginning of value CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 221e4b8e98e8 syneblock/quorum-maker:2.1.0_2.5 "./start_continent..." 16 seconds ago Up 14 seconds 0.0.0.0:22000-22005->22000-22005/tcp, 0.0.0.0:22001->22001/udp continent cad60dec9b0b hello-world "/hello" 50 minutes ago Exited (0) 50 minutes ago

the Node wich is trying to connect: docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES eac1cacf17d6 syneblock/quorum-maker:2.1.0_2.5 "node/pre_start_ch..." 7 seconds ago Up 7 seconds youthful_mirzakhani f0d4b587e16e ubuntu "/bin/bash" About an hour ago Exited (0) 42 minutes ago webserver 70d58c52c832 hello-world "/hello" 24 hours ago Exited (0) 24 hours ago condescending_dijkstra

(I'm sorry for the layout..)

When I do "netstat -pnat | grep listen", I get "(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)" so I ran it with sudo, then I get no output at all...

karthik137 commented 5 years ago

Sorry it was my fault listen should be in caps $ netstat -pnat | grep LISTEN

Looks like your node manager is down. Do one thing. Start the second machine(Node which is trying to connect) by executing start.sh script. Then do $ docker exec -it {container_id} bash

root@{container_id}: /home/node/ root@{container_id}: /home/node/qdata# cd qdata

root@{container_id}: /home/node/qdata#

You should be able to see geth.ipc in qdata directory. If it is there then execute the following command.

root@{container_id}: /home/node/qdata# geth attach geth.ipc

admin.addPeer({enodeAddress of other machine});

Then check your peers from the geth console.

admin.peers

karthik137 commented 5 years ago

You can get enode address from quorum maker dashboard. Go to the first aws instance dashboard -->

http://{your_first_AWS_IP}:22004

click on active nodes tab and check the enode address.

karthik137 commented 5 years ago

One more thing... Did you add rule for NodeManager Port and Constellation port?. If rule is not added then it will never connect. Go to AWS Dashboard and add rules for your ports. Add inbound and outbound rule for the ports.

YouNNN commented 5 years ago

Okay so I did the "netstat -pnat | grep LISTEN: Node which is hosting: tcp 0 0 127.0.0.1:25 0.0.0.0: LISTEN -
tcp 0 0 0.0.0.0:3389 0.0.0.0:
LISTEN -
tcp 0 0 0.0.0.0:111 0.0.0.0: LISTEN -
tcp 0 0 127.0.0.1:3350 0.0.0.0:
LISTEN -
tcp 0 0 0.0.0.0:22 0.0.0.0: LISTEN -
tcp 0 0 127.0.0.1:631 0.0.0.0:
LISTEN -
tcp6 0 0 ::1:25 ::: LISTEN -
tcp6 0 0 :::111 :::
LISTEN -
tcp6 0 0 :::22000 ::: LISTEN -
tcp6 0 0 :::22001 :::
LISTEN -
tcp6 0 0 :::22002 ::: LISTEN -
tcp6 0 0 :::22003 :::
LISTEN -
tcp6 0 0 :::22004 ::: LISTEN -
tcp6 0 0 :::22005 :::
LISTEN -
tcp6 0 0 :::22 ::: LISTEN -
tcp6 0 0 ::1:631 :::
LISTEN -

and Node which is trying to connect: tcp 0 0 127.0.0.1:25 0.0.0.0: LISTEN 1249/master
tcp 0 0 0.0.0.0:3389 0.0.0.0:
LISTEN 1302/xrdp
tcp 0 0 0.0.0.0:111 0.0.0.0: LISTEN 730/rpcbind
tcp 0 0 127.0.0.1:3350 0.0.0.0:
LISTEN 1301/xrdp-sesman
tcp 0 0 0.0.0.0:22 0.0.0.0: LISTEN 1312/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:
LISTEN 1085/cupsd
tcp6 0 0 ::1:25 ::: LISTEN 1249/master
tcp6 0 0 :::111 :::
LISTEN 730/rpcbind
tcp6 0 0 :::22 ::: LISTEN 1312/sshd
tcp6 0 0 ::1:631 :::
LISTEN 1085/cupsd

YouNNN commented 5 years ago

One more thing... Did you add rule for NodeManager Port and Constellation port?. If rule is not added then it will never connect. Go to AWS Dashboard and add rules for your ports. Add inbound and outbound rule for the ports.

I did that manually through the centOS with -p port:port when I set up the Node, if thats what you mean. Therefor I edited the setup.sh: docker run -p 22000:22000 -p 22001:23001 -p 22002:22002 -p 22003:22003 -p 22004:22004 -p 22005:22005 .......

Sorry it was my fault listen should be in caps $ netstat -pnat | grep LISTEN

Looks like your node manager is down. Do one thing. Start the second machine(Node which is trying to connect) by executing start.sh script. Then do $ docker exec -it {container_id} bash

root@{container_id}: /home/node/ root@{container_id}: /home/node/qdata# cd qdata

root@{container_id}: /home/node/qdata#

You should be able to see geth.ipc in qdata directory. If it is there then execute the following command.

root@{container_id}: /home/node/qdata# geth attach geth.ipc

admin.addPeer({enodeAddress of other machine});

Then check your peers from the geth console.

admin.peers

Unfortunately there is no geth.ipc... only directory geth, keystore and logs

karthik137 commented 5 years ago

I did that manually through the centOS with -p port:port when I set up the Node, if thats what you mean. Therefor I edited the setup.sh: docker run -p 23000:23000 -p 23001:23001 -p 23002:23002 -p 23003:23003 -p 23004:23004 -p 23005:23005 .......

No i am not talking about docker. See your first machine works fine because it is not connecting to any other machine. Your second machine will first send join request to first machine then it will start downloading transactions...

I did that manually through the centOS with -p port:port when I set up the Node

Did you setup ec2 machines on your own?. Please refer this [(https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/authorizing-access-to-an-instance.html)]

If you did not setup the ec2 instance then ask your administrator to add inbound and outbound rules for the ports. Its not docker thing. It is firewall issue.

You have to add the rule from amazon ec2 dashboard. Please contact your EC2 administrator or whoever had setup ec2 instances for you....

In your case ports which are used for quorum maker does not accept connections from outside(firewall issue).

Unfortunately there is no geth.ipc... only directory geth, keystore and logs

Yes it won't come up in your second machine because its join request was not accepted due to firewall issues.

YouNNN commented 5 years ago

You are completly right, I didn't set up these instances and I thought the same and wrote an email to the administrator. I hope he can add these rules. Thank you very much for you help that far, I'll wait for the rules to be set and try again. :)

karthik137 commented 5 years ago

Ok. :)

karthik137 commented 5 years ago

Hey did it work?.

YouNNN commented 5 years ago

Hey I'm still waiting for the admin to open the ports...

YouNNN commented 5 years ago

It worked perfectly! Thank you very much!

karthik137 commented 5 years ago

Ok .... Welcome :)