LiskArchive / lisk-sdk

🔩 Lisk software development kit
https://lisk.com
Apache License 2.0
2.72k stars 454 forks source link

Node tries to sync with itself #336

Closed ghost closed 7 years ago

ghost commented 7 years ago

I've noticed that after some time after starting 0.5.0. Node stucks on getting blocks from itself. Possible solutions:

[inf] 2016-12-03 10:18:30 | Broadhash consensus now 1 %
[inf] 2016-12-03 10:18:43 | Finished sync
[inf] 2016-12-03 10:18:54 | Starting sync
[WRN] 2016-12-03 10:18:54 | EVERSION:0.4.1 Removing peer 52.58.156.220:8000 GET /blocks/common?ids=13088404568012503670,6892891772145182357,8341960644728909029,1134575201568741266,14928740171326799900,2786663189856285424,13658550407518916215
[inf] 2016-12-03 10:19:00 | Broadhash consensus now 1 %
[inf] 2016-12-03 10:19:00 | Looking for common block with: PUBLIC_IP_OF_MY_NODE
[WRN] 2016-12-03 10:19:01 | EVERSION:0.3.2 Removing peer 111.207.165.188:8000 GET /list
[inf] 2016-12-03 10:19:03 | Found common block: 6817700788986749089 with: PUBLIC_IP_OF_MY_NODE
[inf] 2016-12-03 10:19:03 | Loading blocks from: PUBLIC_IP_OF_MY_NODE
[inf] 2016-12-03 10:19:03 | Broadhash consensus now 1 %
[inf] 2016-12-03 10:19:15 | Finished sync
[inf] 2016-12-03 10:19:25 | Starting sync
[inf] 2016-12-03 10:19:32 | Broadhash consensus now 1 %
[inf] 2016-12-03 10:19:32 | Looking for common block with: PUBLIC_IP_OF_MY_NODE
[inf] 2016-12-03 10:19:36 | Found common block: 6817700788986749089 with: PUBLIC_IP_OF_MY_NODE
[inf] 2016-12-03 10:19:36 | Loading blocks from: PUBLIC_IP_OF_MY_NODE
[inf] 2016-12-03 10:19:36 | Broadhash consensus now 1 %

Restarting node fix problems temporary, usually for not so long.

Isabello commented 7 years ago

We could address this here,

https://github.com/LiskHQ/lisk/blob/ccd58052aa33148ab2652d97e0a85edcf8ffd016/modules/loader.js#L403 var peer = network.peers[Math.floor(Math.random() * network.peers.length)];

But instead we would probably want to have some validation that the IP is not our own. The best place to address this would instead be here:

https://github.com/LiskHQ/lisk/blob/ccd58052aa33148ab2652d97e0a85edcf8ffd016/modules/peers.js#L245

Peers.prototype.acceptable = function (peers) {
    return _.chain(peers).filter(function (peer) {
        // Removing peers with private ip address
        return !ip.isPrivate(peer.ip);
    }).uniqWith(function (a, b) {
        // Removing non-unique peers
        return (a.ip + a.port) === (b.ip + b.port);
        // Slicing peers up to maxPeers
    }).slice(0, constants.maxPeers).value();
};

We could add logic to remove peers that match our IP address to prevent the peer from becoming a valid selection target and to be put on the DB table.

ghost commented 7 years ago

This fix does not work. I have noticed in 0.6.0 that issue remains. Reloading lisk resolved issue.

[WRN] 2017-03-12 16:17:30 | ETIMEOUT Ban 83.8.60.99:8000 1 minutes GET http://83.8.60.99:8000/peer/height
[WRN] 2017-03-12 16:17:30 | ETIMEOUT Ban 85.76.68.167:8000 1 minutes GET http://85.76.68.167:8000/peer/height
[WRN] 2017-03-12 16:17:30 | ETIMEOUT Ban 52.40.182.224:8000 1 minutes GET http://52.40.182.224:8000/peer/height
[WRN] 2017-03-12 16:17:30 | ETIMEOUT Ban 83.25.156.28:8000 1 minutes GET http://83.25.156.28:8000/peer/height
[WRN] 2017-03-12 16:17:30 | ETIMEOUT Ban 45.32.190.145:8000 1 minutes GET http://45.32.190.145:8000/peer/height
[WRN] 2017-03-12 16:17:30 | ETIMEOUT Ban 83.25.138.48:8000 1 minutes GET http://83.25.138.48:8000/peer/height
[WRN] 2017-03-12 16:17:30 | ETIMEOUT Ban 52.199.12.246:8000 1 minutes GET http://52.199.12.246:8000/peer/height
[WRN] 2017-03-12 16:17:30 | ETIMEOUT Ban 213.136.88.142:8000 1 minutes GET http://213.136.88.142:8000/peer/height
[WRN] 2017-03-12 16:17:30 | ETIMEOUT Ban 5.102.51.34:8000 1 minutes GET http://5.102.51.34:8000/peer/height
[WRN] 2017-03-12 16:17:30 | ETIMEOUT Ban 77.174.128.147:8000 1 minutes GET http://77.174.128.147:8000/peer/height
[inf] 2017-03-12 16:17:32 | Found common block: 6786925477605855267 with: PUBLIC_IP_OF_THIS_NODE:8000
[inf] 2017-03-12 16:17:32 | Loading blocks from: PUBLIC_IP_OF_THIS_NODE:8000
[inf] 2017-03-12 16:17:32 | Broadhash consensus now 1 %
[inf] 2017-03-12 16:17:34 | Finished sync
[WRN] 2017-03-12 16:17:35 | EUNAVAILABLE Removing peer 52.160.98.183:8000 GET http://52.160.98.183:8000/peer/height
[WRN] 2017-03-12 16:17:35 | EUNAVAILABLE Removing peer 13.89.42.130:8000 GET http://13.89.42.130:8000/peer/height
[WRN] 2017-03-12 16:17:35 | EUNAVAILABLE Removing peer 119.253.64.45:8000 GET http://119.253.64.45:8000/peer/height
[WRN] 2017-03-12 16:17:35 | EUNAVAILABLE Removing peer 138.201.19.234:8000 GET http://138.201.19.234:8000/peer/height
[WRN] 2017-03-12 16:17:35 | EUNAVAILABLE Removing peer 5.9.70.174:8000 GET http://5.9.70.174:8000/peer/height
[WRN] 2017-03-12 16:17:35 | ERESPONSE 404 Removing peer 71.197.2.119:8000 GET http://71.197.2.119:8000/peer/height
[WRN] 2017-03-12 16:17:35 | ERESPONSE 404 Removing peer 46.228.6.34:8000 GET http://46.228.6.34:8000/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 52.198.118.74:8000 1 minutes GET http://52.198.118.74:8000/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 84.147.157.156:8010 1 minutes GET http://84.147.157.156:8010/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 96.127.136.18:8000 1 minutes GET http://96.127.136.18:8000/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 159.203.69.196:8000 1 minutes GET http://159.203.69.196:8000/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 166.137.177.29:8000 1 minutes GET http://166.137.177.29:8000/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 46.28.204.68:8000 1 minutes GET http://46.28.204.68:8000/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 45.76.223.64:8000 1 minutes GET http://45.76.223.64:8000/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 83.25.139.91:8000 1 minutes GET http://83.25.139.91:8000/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 84.147.146.219:8010 1 minutes GET http://84.147.146.219:8010/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 99.73.224.185:8000 1 minutes GET http://99.73.224.185:8000/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 83.20.50.141:8000 1 minutes GET http://83.20.50.141:8000/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 95.235.111.154:8000 1 minutes GET http://95.235.111.154:8000/peer/height
[WRN] 2017-03-12 16:17:40 | ETIMEOUT Ban 83.23.50.32:8000 1 minutes GET http://83.23.50.32:8000/peer/height
[inf] 2017-03-12 16:17:43 | Starting sync
[inf] 2017-03-12 16:17:45 | Broadhash consensus now 1 %
[inf] 2017-03-12 16:17:45 | Looking for common block with: PUBLIC_IP_OF_THIS_NODE:8000
[WRN] 2017-03-12 16:17:45 | EUNAVAILABLE Removing peer 52.160.98.183:8000 GET http://52.160.98.183:8000/peer/height
[WRN] 2017-03-12 16:17:45 | EUNAVAILABLE Removing peer 13.89.42.130:8000 GET http://13.89.42.130:8000/peer/height
[WRN] 2017-03-12 16:17:45 | EUNAVAILABLE Removing peer 119.253.64.45:8000 GET http://119.253.64.45:8000/peer/height
[WRN] 2017-03-12 16:17:45 | EUNAVAILABLE Removing peer 138.201.19.234:8000 GET http://138.201.19.234:8000/peer/height
[WRN] 2017-03-12 16:17:45 | ERESPONSE 404 Removing peer 71.197.2.119:8000 GET http://71.197.2.119:8000/peer/height
[WRN] 2017-03-12 16:17:46 | ERESPONSE 404 Removing peer 46.228.6.34:8000 GET http://46.228.6.34:8000/peer/height
[inf] 2017-03-12 16:17:49 | Found common block: 6786925477605855267 with: PUBLIC_IP_OF_THIS_NODE:8000
[inf] 2017-03-12 16:17:49 | Loading blocks from: PUBLIC_IP_OF_THIS_NODE:8000
[inf] 2017-03-12 16:17:50 | Broadhash consensus now 1 %
[WRN] 2017-03-12 16:17:50 | ETIMEOUT Ban 50.161.67.160:8000 1 minutes GET http://50.161.67.160:8000/peer/height
[WRN] 2017-03-12 16:17:50 | ETIMEOUT Ban 83.25.158.80:8000 1 minutes GET http://83.25.158.80:8000/peer/height
[WRN] 2017-03-12 16:17:50 | ETIMEOUT Ban 111.207.165.187:8000 1 minutes GET http://111.207.165.187:8000/peer/height
[WRN] 2017-03-12 16:17:50 | ETIMEOUT Ban 139.196.210.16:8000 1 minutes GET http://139.196.210.16:8000/peer/height
[WRN] 2017-03-12 16:17:50 | ETIMEOUT Ban 83.23.112.69:8000 1 minutes GET http://83.23.112.69:8000/peer/height
[WRN] 2017-03-12 16:17:50 | ETIMEOUT Ban 83.20.65.129:8000 1 minutes GET http://83.20.65.129:8000/peer/height
[WRN] 2017-03-12 16:17:50 | ETIMEOUT Ban 83.21.213.162:8000 1 minutes GET http://83.21.213.162:8000/peer/height
[inf] 2017-03-12 16:17:52 | Finished sync
[inf] 2017-03-12 16:17:54 | Starting sync
[WRN] 2017-03-12 16:17:56 | EUNAVAILABLE Removing peer 52.160.98.183:8000 GET http://52.160.98.183:8000/peer/height
[WRN] 2017-03-12 16:17:56 | EUNAVAILABLE Removing peer 13.89.42.130:8000 GET http://13.89.42.130:8000/peer/height
[WRN] 2017-03-12 16:17:56 | EUNAVAILABLE Removing peer 5.9.70.174:8000 GET http://5.9.70.174:8000/peer/height
[WRN] 2017-03-12 16:17:56 | EUNAVAILABLE Removing peer 119.253.64.45:8000 GET http://119.253.64.45:8000/peer/height
[WRN] 2017-03-12 16:17:56 | EUNAVAILABLE Removing peer 138.201.19.234:8000 GET http://138.201.19.234:8000/peer/height
[WRN] 2017-03-12 16:17:56 | ERESPONSE 404 Removing peer 71.197.2.119:8000 GET http://71.197.2.119:8000/peer/height
[WRN] 2017-03-12 16:17:56 | ERESPONSE 500 Removing peer 5.189.174.126:8000 GET http://5.189.174.126:8000/peer/height
[WRN] 2017-03-12 16:17:56 | ERESPONSE 404 Removing peer 46.228.6.34:8000 GET http://46.228.6.34:8000/peer/height
[inf] 2017-03-12 16:17:57 | Broadhash consensus now 1 %
[inf] 2017-03-12 16:17:57 | Looking for common block with: PUBLIC_IP_OF_THIS_NODE:8000
[WRN] 2017-03-12 16:18:00 | ETIMEOUT Ban 145.101.25.168:8000 1 minutes GET http://145.101.25.168:8000/peer/height
[WRN] 2017-03-12 16:18:00 | ETIMEOUT Ban 83.25.154.33:8000 1 minutes GET http://83.25.154.33:8000/peer/height
[WRN] 2017-03-12 16:18:00 | ETIMEOUT Ban 37.120.189.9:8000 1 minutes GET http://37.120.189.9:8000/peer/height
[WRN] 2017-03-12 16:18:00 | ETIMEOUT Ban 104.155.104.146:8000 1 minutes GET http://104.155.104.146:8000/peer/height
[WRN] 2017-03-12 16:18:00 | ETIMEOUT Ban 83.21.212.188:8000 1 minutes GET http://83.21.212.188:8000/peer/height
[WRN] 2017-03-12 16:18:00 | ETIMEOUT Ban 83.23.59.46:8000 1 minutes GET http://83.23.59.46:8000/peer/height
[WRN] 2017-03-12 16:18:00 | ETIMEOUT Ban 83.21.208.41:8000 1 minutes GET http://83.21.208.41:8000/peer/height
[WRN] 2017-03-12 16:18:00 | ETIMEOUT Ban 83.23.181.83:8000 1 minutes GET http://83.23.181.83:8000/peer/height
[WRN] 2017-03-12 16:18:00 | ETIMEOUT Ban 209.181.66.82:8000 1 minutes GET http://209.181.66.82:8000/peer/height
[WRN] 2017-03-12 16:18:00 | ETIMEOUT Ban 83.25.145.86:8000 1 minutes GET http://83.25.145.86:8000/peer/height
[WRN] 2017-03-12 16:18:00 | ETIMEOUT Ban 77.248.48.178:8000 1 minutes GET http://77.248.48.178:8000/peer/height
[WRN] 2017-03-12 16:18:00 | ETIMEOUT Ban 93.238.41.78:8010 1 minutes GET http://93.238.41.78:8010/peer/height
[inf] 2017-03-12 16:18:02 | Found common block: 6786925477605855267 with: PUBLIC_IP_OF_THIS_NODE:8000
[inf] 2017-03-12 16:18:02 | Loading blocks from: PUBLIC_IP_OF_THIS_NODE:8000
[inf] 2017-03-12 16:18:02 | Broadhash consensus now 1 %
[inf] 2017-03-12 16:18:04 | Finished sync
[WRN] 2017-03-12 16:18:06 | EUNAVAILABLE Removing peer 52.160.98.183:8000 GET http://52.160.98.183:8000/peer/height
[WRN] 2017-03-12 16:18:06 | EUNAVAILABLE Removing peer 13.89.42.130:8000 GET http://13.89.42.130:8000/peer/height
[WRN] 2017-03-12 16:18:06 | EUNAVAILABLE Removing peer 119.253.64.45:8000 GET http://119.253.64.45:8000/peer/height
[WRN] 2017-03-12 16:18:06 | EUNAVAILABLE Removing peer 138.201.19.234:8000 GET http://138.201.19.234:8000/peer/height
[WRN] 2017-03-12 16:18:06 | ERESPONSE 404 Removing peer 71.197.2.119:8000 GET http://71.197.2.119:8000/peer/height
[WRN] 2017-03-12 16:18:06 | ERESPONSE 404 Removing peer 46.228.6.34:8000 GET http://46.228.6.34:8000/peer/height
[WRN] 2017-03-12 16:18:11 | ETIMEOUT Ban 84.213.9.92:8000 1 minutes GET http://84.213.9.92:8000/peer/height
[WRN] 2017-03-12 16:18:11 | ETIMEOUT Ban 83.20.63.66:8000 1 minutes GET http://83.20.63.66:8000/peer/height
[WRN] 2017-03-12 16:18:11 | ETIMEOUT Ban 79.186.136.170:8000 1 minutes GET http://79.186.136.170:8000/peer/height
[WRN] 2017-03-12 16:18:11 | ETIMEOUT Ban 83.11.107.221:8000 1 minutes GET http://83.11.107.221:8000/peer/height
[WRN] 2017-03-12 16:18:11 | ETIMEOUT Ban 68.234.183.96:8000 1 minutes GET http://68.234.183.96:8000/peer/height
[WRN] 2017-03-12 16:18:11 | ETIMEOUT Ban 83.20.231.45:8000 1 minutes GET http://83.20.231.45:8000/peer/height
[WRN] 2017-03-12 16:18:11 | ETIMEOUT Ban 67.198.50.210:8000 1 minutes GET http://67.198.50.210:8000/peer/height
[WRN] 2017-03-12 16:18:11 | ETIMEOUT Ban 83.11.106.103:8000 1 minutes GET http://83.11.106.103:8000/peer/height
[WRN] 2017-03-12 16:18:11 | ETIMEOUT Ban 83.23.108.134:8000 1 minutes GET http://83.23.108.134:8000/peer/height
[inf] 2017-03-12 16:18:12 | Starting sync
[inf] 2017-03-12 16:18:13 | Broadhash consensus now 1 %
[inf] 2017-03-12 16:18:13 | Looking for common block with: PUBLIC_IP_OF_THIS_NODE:8000
[WRN] 2017-03-12 16:18:16 | EUNAVAILABLE Removing peer 52.160.98.183:8000 GET http://52.160.98.183:8000/peer/height
[WRN] 2017-03-12 16:18:16 | EUNAVAILABLE Removing peer 13.89.42.130:8000 GET http://13.89.42.130:8000/peer/height
[WRN] 2017-03-12 16:18:16 | EUNAVAILABLE Removing peer 119.253.64.45:8000 GET http://119.253.64.45:8000/peer/height
[WRN] 2017-03-12 16:18:16 | EUNAVAILABLE Removing peer 138.201.19.234:8000 GET http://138.201.19.234:8000/peer/height
[WRN] 2017-03-12 16:18:16 | ERESPONSE 404 Removing peer 71.197.2.119:8000 GET http://71.197.2.119:8000/peer/height
[WRN] 2017-03-12 16:18:16 | EUNAVAILABLE Removing peer 5.9.70.174:8000 GET http://5.9.70.174:8000/peer/height
[WRN] 2017-03-12 16:18:16 | ERESPONSE 404 Removing peer 46.228.6.34:8000 GET http://46.228.6.34:8000/peer/height
[inf] 2017-03-12 16:18:18 | Found common block: 6786925477605855267 with: PUBLIC_IP_OF_THIS_NODE:8000
[inf] 2017-03-12 16:18:18 | Loading blocks from: PUBLIC_IP_OF_THIS_NODE:8000
[inf] 2017-03-12 16:18:18 | Broadhash consensus now 1 %
[inf] 2017-03-12 16:18:21 | Finished sync
[WRN] 2017-03-12 16:18:21 | ETIMEOUT Ban 54.187.22.60:8000 1 minutes GET http://54.187.22.60:8000/peer/height
[WRN] 2017-03-12 16:18:21 | ETIMEOUT Ban 83.21.210.16:8000 1 minutes GET http://83.21.210.16:8000/peer/height
[WRN] 2017-03-12 16:18:21 | ETIMEOUT Ban 83.23.59.201:8000 1 minutes GET http://83.23.59.201:8000/peer/height
[WRN] 2017-03-12 16:18:21 | ETIMEOUT Ban 83.20.14.145:8000 1 minutes GET http://83.20.14.145:8000/peer/height
[WRN] 2017-03-12 16:18:21 | ETIMEOUT Ban 83.25.158.31:8000 1 minutes GET http://83.25.158.31:8000/peer/height