search

Chia-Network / chia-blockchain

Chia blockchain python implementation (full node, farmer, harvester, timelord, and wallet)
Apache License 2.0
10.82k stars 2.03k forks source link

[Bug] 1.3.4 harvester peer disconnected #11327

Closed tonyx888 closed 2 years ago

tonyx888 commented 2 years ago

What happened?

Hello everyone, since I updated to version 1.3.4 I have many problems with harvesters, I have a network of 26 harvesters and 1 farmer all updated to version 1.3.4 Unfortunately my internet connection every day falls 1 or 2 times, before with 1.2.11 the harvesters reconnected immediately, now with the latest version of the harvesters on the log I have this error and loses the connection forcing me to have to manually restart the harvesters.

2022-04-26T16:34:57.006 harvester harvester_server : INFO Cannot connect to host 192.168.0.5:8447 ssl:<ssl.SSLContext object at 0x000001A4029977C0> [Periodo di timeout del semaforo scaduto] 2022-04-26T16:35:00.021 harvester harvester : INFO Reconnecting to peer {'host': '192.168.0.5', 'port': 8447} 2022-04-26T16:35:07.113 harvester harvester_server : INFO Connected with farmer {'host': '192.168.0.5', 'port': 8447} 2022-04-26T16:35:07.238 harvester chia.plotting.manager : INFO Loaded 534096 bytes of cached data 2022-04-26T16:35:35.372 harvester chia.plot_sync.sender : INFO _send_next_message didn't receive response expected_message_type: plot_sync_loaded, expected_identifier: {'message_id': 7, 'sync_id': 1650983707, 'timestamp': 1650983725}, message None 2022-04-26T16:35:55.446 harvester chia.plot_sync.sender : INFO _send_next_message didn't receive response expected_message_type: plot_sync_loaded, expected_identifier: {'message_id': 7, 'sync_id': 1650983707, 'timestamp': 1650983745}, message None 2022-04-26T16:36:15.504 harvester chia.plot_sync.sender : INFO _send_next_message didn't receive response expected_message_type: plot_sync_loaded, expected_identifier: {'message_id': 7, 'sync_id': 1650983707, 'timestamp': 1650983765}, message None 2022-04-26T16:36:35.577 harvester chia.plot_sync.sender : INFO _send_next_message didn't receive response expected_message_type: plot_sync_loaded, expected_identifier: {'message_id': 7, 'sync_id': 1650983707, 'timestamp': 1650983785}, message None 2022-04-26T16:36:55.651 harvester chia.plot_sync.sender : INFO _send_next_message didn't receive response expected_message_type: plot_sync_loaded, expected_identifier: {'message_id': 7, 'sync_id': 1650983707, 'timestamp': 1650983805}, message None 2022-04-26T16:37:15.708 harvester chia.plot_sync.sender : INFO _send_next_message didn't receive response expected_message_type: plot_sync_loaded, expected_identifier: {'message_id': 7, 'sync_id': 1650983707, 'timestamp': 1650983825}, message None 2022-04-26T16:37:25.269 harvester harvester_server : INFO Connection closed: 192.168.0.5, node id: 098ba0fa3b705a6f8084d5c5869ac64c7c569fd2820a13a9421b15d2d1978c23 2022-04-26T16:37:25.269 harvester chia.harvester.harvester: INFO peer disconnected {'host': '192.168.0.5', 'port': 8447}

PLEASE HELP ME Thanks all

Version

1.3.4

What platform are you using?

Windows

What ui mode are you using?

CLI

Relevant log output

2022-04-26T16:34:57.006 harvester harvester_server : INFO Cannot connect to host 192.168.0.5:8447 ssl:<ssl.SSLContext object at 0x000001A4029977C0> [Periodo di timeout del semaforo scaduto]
2022-04-26T16:35:00.021 harvester harvester : INFO Reconnecting to peer {'host': '192.168.0.5', 'port': 8447}
2022-04-26T16:35:07.113 harvester harvester_server : INFO Connected with farmer {'host': '192.168.0.5', 'port': 8447}
2022-04-26T16:35:07.238 harvester chia.plotting.manager : INFO Loaded 534096 bytes of cached data
2022-04-26T16:35:35.372 harvester chia.plot_sync.sender : INFO _send_next_message didn't receive response expected_message_type: plot_sync_loaded, expected_identifier: {'message_id': 7, 'sync_id': 1650983707, 'timestamp': 1650983725}, message None
2022-04-26T16:35:55.446 harvester chia.plot_sync.sender : INFO _send_next_message didn't receive response expected_message_type: plot_sync_loaded, expected_identifier: {'message_id': 7, 'sync_id': 1650983707, 'timestamp': 1650983745}, message None
2022-04-26T16:36:15.504 harvester chia.plot_sync.sender : INFO _send_next_message didn't receive response expected_message_type: plot_sync_loaded, expected_identifier: {'message_id': 7, 'sync_id': 1650983707, 'timestamp': 1650983765}, message None
2022-04-26T16:36:35.577 harvester chia.plot_sync.sender : INFO _send_next_message didn't receive response expected_message_type: plot_sync_loaded, expected_identifier: {'message_id': 7, 'sync_id': 1650983707, 'timestamp': 1650983785}, message None
2022-04-26T16:36:55.651 harvester chia.plot_sync.sender : INFO _send_next_message didn't receive response expected_message_type: plot_sync_loaded, expected_identifier: {'message_id': 7, 'sync_id': 1650983707, 'timestamp': 1650983805}, message None
2022-04-26T16:37:15.708 harvester chia.plot_sync.sender : INFO _send_next_message didn't receive response expected_message_type: plot_sync_loaded, expected_identifier: {'message_id': 7, 'sync_id': 1650983707, 'timestamp': 1650983825}, message None
2022-04-26T16:37:25.269 harvester harvester_server : INFO Connection closed: 192.168.0.5, node id: 098ba0fa3b705a6f8084d5c5869ac64c7c569fd2820a13a9421b15d2d1978c23
2022-04-26T16:37:25.269 harvester chia.harvester.harvester: INFO peer disconnected {'host': '192.168.0.5', 'port': 8447}
ferdiina commented 2 years ago

the same issue . i have to restart the daemon to fix. restart harvester invalid. but i found the windows client working fine. in my centos7/8 and ubuntu 20.04.4 happed in amost same time

xdustinface commented 2 years ago

@tonyx888 Can you post the log from the farmer also?

tonyx888 commented 2 years ago

farmer log at same time

ValueError: Failed to fetch block 1891178 from {'host': '176.240.176.13', 'port': 8444}, timed out

2022-04-26T13:08:53.424 full_node chia.full_node.full_node: WARNING Block validation time: 2.48 seconds, pre_validation time: 0.22 seconds, cost: 4188543779, percent full: 38.078% header_hash: 0e06e7e2ea41f472eaf7f4e8f67cf06a73685e3049282f58f3417e751d4efc19 height: 1891423 2022-04-26T14:36:27.772 full_node chia.full_node.full_node: WARNING Block validation time: 2.34 seconds, pre_validation time: 0.22 seconds, cost: 4206945268, percent full: 38.245% header_hash: 1843b9e51e68baf827d6d97bfb45b1211ca9ff53b9b50cb3374a9b9eaf17172f height: 1891690 2022-04-26T14:58:01.184 full_node chia.full_node.full_node: WARNING Block validation time: 2.60 seconds, pre_validation time: 0.23 seconds, cost: 5090240954, percent full: 46.275% header_hash: 90090c125d32f5376b43802bf20c5971afb35384186f2fe89e93b09413c10b72 height: 1891752 2022-04-26T15:54:04.170 full_node chia.full_node.full_node: WARNING Block validation time: 3.10 seconds, pre_validation time: 1.51 seconds, cost: 5110249221, percent full: 46.457% header_hash: 2003215797951c210cc050096f48c51a14279b5efc1addbaad812c95b1a89041 height: 1891919 2022-04-26T16:09:11.710 farmer chia.farmer.farmer : ERROR Exception in GET /farmer https://eu1.pool.space, Cannot connect to host eu1.pool.space:443 ssl:<ssl.SSLContext object at 0x000001D0FBD444C0> [getaddrinfo failed] 2022-04-26T16:09:22.812 farmer chia.farmer.farmer : ERROR Exception in GET /farmer https://eu1.pool.space, Cannot connect to host eu1.pool.space:443 ssl:<ssl.SSLContext object at 0x000001D0F61FAF40> [getaddrinfo failed] 2022-04-26T16:14:40.954 farmer chia.farmer.farmer : ERROR Exception in GET /farmer https://eu1.pool.space, Cannot connect to host eu1.pool.space:443 ssl:<ssl.SSLContext object at 0x000001D0FD46B9C0> [getaddrinfo failed] 2022-04-26T16:14:40.986 farmer chia.farmer.farmer : ERROR Exception in GET /farmer https://eu1.pool.space, Cannot connect to host eu1.pool.space:443 ssl:<ssl.SSLContext object at 0x000001D0FD46B6C0> [getaddrinfo failed] 2022-04-26T16:19:48.297 farmer chia.farmer.farmer : ERROR Exception in GET /farmer https://eu1.pool.space, Cannot connect to host eu1.pool.space:443 ssl:<ssl.SSLContext object at 0x000001D0FD46BF40> [getaddrinfo failed] 2022-04-26T16:19:48.329 farmer chia.farmer.farmer : ERROR Exception in GET /farmer https://eu1.pool.space, Cannot connect to host eu1.pool.space:443 ssl:<ssl.SSLContext object at 0x000001D0FD46B6C0> [getaddrinfo failed] 2022-04-26T16:19:53.414 full_node chia.full_node.full_node: WARNING querying DNS introducer failed: The resolution lifetime expired after 30.014343738555908 seconds: Server 8.8.8.8 UDP port 53 answered The DNS operation timed out after 1.9993996620178223 seconds; Server 8.8.4.4 UDP port 53 answered The DNS operation timed out after 1.9993236064910889 seconds; Server 8.8.8.8 UDP port 53 answered The DNS operation timed out after 2.0 seconds; Server 8.8.4.4 UDP port 53 answered The DNS operation timed out after 1.999326467514038 seconds; Server 8.8.8.8 UDP port 53 answered The DNS operation timed out after 2.0 seconds; Server 8.8.4.4 UDP port 53 answered The DNS operation timed out after 1.9993302822113037 seconds; Server 8.8.8.8 UDP port 53 answered The DNS operation timed out after 1.999305248260498 seconds; Server 8.8.4.4 UDP port 53 answered The DNS operation timed out after 1.9993219375610352 seconds; Server 8.8.8.8 UDP port 53 answered The DNS operation timed out after 1.9993634223937988 seconds; Server 8.8.4.4 UDP port 53 answered The DNS operation timed out after 1.9993104934692383 seconds; Server 8.8.8.8 UDP port 53 answered The DNS operation timed out after 2.0 seconds; Server 8.8.4.4 UDP port 53 answered The DNS operation timed out after 1.9993312358856201 seconds; Server 8.8.8.8 UDP port 53 answered The DNS operation timed out after 0.7650387287139893 seconds 2022-04-26T16:23:03.588 farmer chia.farmer.farmer : ERROR Exception in GET /pool_info https://eu1.pool.space, Cannot connect to host eu1.pool.space:443 ssl:<ssl.SSLContext object at 0x000001D0FBD444C0> [getaddrinfo failed] 2022-04-26T16:23:03.619 farmer chia.farmer.farmer : ERROR Exception in GET /pool_info https://eu1.pool.space, Cannot connect to host eu1.pool.space:443 ssl:<ssl.SSLContext object at 0x000001D0FBD44D40> [getaddrinfo failed] 2022-04-26T16:25:17.600 farmer chia.farmer.farmer : ERROR Exception in GET /farmer https://eu1.pool.space, Cannot connect to host eu1.pool.space:443 ssl:<ssl.SSLContext object at 0x000001D0FD46B540> [getaddrinfo failed] 2022-04-26T16:25:17.633 farmer chia.farmer.farmer : ERROR Exception in GET /farmer https://eu1.pool.space, Cannot connect to host eu1.pool.space:443 ssl:<ssl.SSLContext object at 0x000001D0FD46BAC0> [getaddrinfo failed] 2022-04-26T16:30:24.801 farmer chia.farmer.farmer : ERROR Exception in GET /farmer https://eu1.pool.space, Cannot connect to host eu1.pool.space:443 ssl:<ssl.SSLContext object at 0x000001D0FD46BEC0> [getaddrinfo failed] 2022-04-26T16:30:24.833 farmer chia.farmer.farmer : ERROR Exception in GET /farmer https://eu1.pool.space, Cannot connect to host eu1.pool.space:443 ssl:<ssl.SSLContext object at 0x000001D0FD46BAC0> [getaddrinfo failed] 2022-04-26T16:45:55.799 daemon main : ERROR Unexpected exception trying to send to websocket: Cannot write to closing transport Traceback (most recent call last): File "Lib\site-packages\chia\daemon\server.py", line 239, in incoming_connection File "aiohttp\web_ws.py", line 300, in send_str File "aiohttp\http_websocket.py", line 687, in send File "aiohttp\http_websocket.py", line 650, in _send_frame File "aiohttp\http_websocket.py", line 660, in _write ConnectionResetError: Cannot write to closing transport

tonyx888 commented 2 years ago

in the end I decided to switch to the hpool PP client without using chia

pzstarnet commented 2 years ago

I have a problem too，I have a network of 11 harvesters and 1 main server all updated to version 1.3.4，All servers are windows，The main server has an error communicating with the harvester and cannot function properly.

wallentx commented 2 years ago

This appears to duplicate https://github.com/Chia-Network/chia-blockchain/issues/11056 We are will continue to investigate this.

emlowe commented 2 years ago

Fixed in 1.4.0