search

Chia-Network / chia-blockchain

Chia blockchain python implementation (full node, farmer, harvester, timelord, and wallet)
Apache License 2.0
10.83k stars 2.02k forks source link

[Bug] Closing GUI while leaving pool puts Plot NFT in the Invalid state #18420

Open chia-rschmidt opened 1 month ago

chia-rschmidt commented 1 month ago

What happened?

When attempting to leave a pool and revert to self pooling, I waited over 30 minutes and then accidentally closed the GUI. After reopening the GUI, the Plot NFT was in the Invalid state and alerted that I couldn't attempt to leave the pool again until a specific height.

TC-Elderberry TC-Elderberry2

Version

2.4.3-rc1

What platform are you using?

Linux

What ui mode are you using?

GUI

Relevant log output

2024-08-04T14:26:34.803 wallet chia.wallet.wallet         : WARNING  TransactionRecord SpendBundle ID: b'\xbb\xc6\xe3%\xfeBb\x1c\x1eX\xaf!m\xdcg\x08\x0e\xbd\x1b\xd8qUO\xbc\xda\x1c\xfe\xc3\xcc\xde=\xff' not in mempool. (peer, included, error) list: []
2024-08-04T14:27:34.789 farmer chia.farmer.farmer         : WARNING  No pool specific difficulty has been set for 0c58a2da1fd0bd1661032751312d8d7835d8e5d302147d54e5f5f19af2cb3637, check communication with the pool, skipping this signage point, pool: https://txch.spacefarmers.io 
2024-08-04T14:27:35.200 farmer chia.pools.pool_config     : ERROR    Failed to update pool config entry for launcher_id 397837e7ad51b898e20364da60fd23aa3aa8ccff0b33f162358fdabdfc5b3c0f
2024-08-04T14:27:36.195 farmer chia.farmer.farmer         : WARNING  GET /farmer response: {'error_code': 10, 'error_message': 'Farmer with launcher_id 397837e7ad51b898e20364da60fd23aa3aa8ccff0b33f162358fdabdfc5b3c0f unknown.'}
2024-08-04T14:32:20.005 wallet chia.wallet.wallet         : WARNING  TransactionRecord SpendBundle ID: b'c8\xdcZ\xf5+\xdc-\x11\x83S)\xae\r\xc8\xf6\x1d\xb5l\xbf\xd2?\xab{uY\x02\t\xffJ\xc5\x1c' not in mempool. (peer, included, error) list: []
2024-08-04T14:32:20.021 wallet chia.rpc.rpc_server        : WARNING  Error while handling message: Traceback (most recent call last):
  File "chia/rpc/rpc_server.py", line 340, in safe_handle
  File "chia/rpc/rpc_server.py", line 331, in ws_api
  File "chia/rpc/wallet_rpc_api.py", line 2543, in did_get_did
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'

2024-08-04T14:32:20.021 wallet chia.rpc.rpc_server        : WARNING  Error while handling message: Traceback (most recent call last):
  File "chia/rpc/rpc_server.py", line 340, in safe_handle
  File "chia/rpc/rpc_server.py", line 331, in ws_api
  File "chia/rpc/wallet_rpc_api.py", line 2211, in did_get_wallet_name
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'NoneType'

2024-08-04T14:33:22.713 wallet chia.pools.pool_puzzles    : ERROR    Unexpected return from PoolWallet Smart Contract code cannot unpack non-iterable int object
2024-08-04T14:34:01.159 wallet wallet_server              : WARNING  API not ready, ignore request: {'data': '0x9a2ae01da85bf639f36e4edc909abec441c5f73e8327ce3944f2a0868be8a5900011844100000000000000000000009f1be4b87200118441',
 'id': None,
 'type': 50}
2024-08-04T14:34:01.766 wallet wallet_server              : ERROR    Caught exception: RuntimeError: wallet state manager not assigned
Traceback (most recent call last):
  File "chia/util/log_exceptions.py", line 20, in log_exceptions
  File "chia/server/ws_connection.py", line 344, in close
  File "chia/server/server.py", line 559, in connection_closed
  File "chia/wallet/wallet_node.py", line 745, in on_disconnect
  File "chia/wallet/wallet_node.py", line 186, in wallet_state_manager
RuntimeError: wallet state manager not assigned

2024-08-04T14:34:04.169 wallet chia.wallet.wallet_node    : WARNING  Non-trusted peer PeerInfo(_ip=IPv4Address('34.215.153.245'), _port=58444) is not synced, disconnecting
2024-08-04T14:34:04.169 wallet wallet_server              : WARNING  Banning 34.215.153.245 for 120 seconds
2024-08-04T14:34:04.316 wallet chia.pools.pool_puzzles    : ERROR    Unexpected return from PoolWallet Smart Contract code cannot unpack non-iterable int object
2024-08-04T14:34:04.471 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0x2abfb4e5f24bcb79a4e1f2136c3591e4299e0f31e83654e498215b4b20715976'}
2024-08-04T14:34:04.474 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0x8c45b008805ab85016138b9a83d3688376f1ba4e7f9670c3a3d692ef1dd90483'}
2024-08-04T14:34:04.476 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0xaf8bf642e9a194902c9a61bf24a86428d8b6ee926f6c492a8908a787e9ccad07'}
2024-08-04T14:34:04.477 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0xbbc6e325fe42621c1e58af216ddc67080ebd1bd871554fbcda1cfec3ccde3dff'}
2024-08-04T14:34:04.479 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0x6338dc5af52bdc2d11835329ae0dc8f61db56cbfd23fab7b75590209ff4ac51c'}
2024-08-04T14:34:04.481 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0x74dff1afa93f28bd2248461c9ab44019c38f71c314c4c581398a8f22df4c110d'}
2024-08-04T14:53:50.966 full_node full_node_server        : ERROR    Exception: Connection lost with 67.150.111.42
2024-08-04T14:53:50.967 full_node full_node_server        : ERROR    Exception Stack: Traceback (most recent call last):
  File "asyncio/selector_events.py", line 862, in _read_ready__data_received
TimeoutError: [Errno 110] Connection timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "chia/server/ws_connection.py", line 374, in outbound_handler
  File "chia/server/ws_connection.py", line 647, in _send_message
  File "aiohttp/web_ws.py", line 342, in send_bytes
  File "aiohttp/http_websocket.py", line 727, in send
  File "aiohttp/http_websocket.py", line 691, in _send_frame
  File "aiohttp/base_protocol.py", line 95, in _drain_helper
ConnectionError: Connection lost

2024-08-04T15:28:13.917 farmer chia.pools.pool_config     : ERROR    Failed to update pool config entry for launcher_id 397837e7ad51b898e20364da60fd23aa3aa8ccff0b33f162358fdabdfc5b3c0f
2024-08-04T15:42:04.909 full_node asyncio                 : ERROR    Future exception was never retrieved
future: <Future finished exception=ConnectionError('Connection lost')>
aiohttp.client_exceptions.ClientOSError: [Errno 104] Connection reset by peer

The above exception was the direct cause of the following exception:

ConnectionError: Connection lost
2024-08-04T16:02:28.337 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'INVALID_FEE_TOO_CLOSE_TO_ZERO',
 'status': 3,
 'txid': '0x7fba61a06ec30edf9751a17bd97b7d07266cd65689b0727b00f5ae85c527aad2'}
2024-08-04T16:09:16.850 wallet chia.wallet.wallet_state_manager: ERROR    Failed to add coin_state: CoinState { coin: Coin { parent_coin_info: fff7747765ab55cbecd0d84e665499ff3c65fda8c16e86960d2b2e8c14dffa7e, puzzle_hash: 3cd225c5ecb2ed63b89d2b1cbeaa91350ea69991e0fc8f4eb937182c903c335c, amount: 6968000 }, spent_height: Some(1148268), created_height: Some(1148028) }, error: list index out of range
Traceback (most recent call last):
  File "chia/wallet/wallet_state_manager.py", line 1747, in _add_coin_states
  File "chia/wallet/trade_manager.py", line 190, in coins_of_interest_farmed
IndexError: list index out of range
2024-08-04T16:19:33.490 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'INVALID_FEE_TOO_CLOSE_TO_ZERO',
 'status': 3,
 'txid': '0x673aa70b35a8dbeff35184de340ab40dc88844b715933d7e851e3612357451b9'}
2024-08-04T16:19:33.495 wallet chia.wallet.wallet         : WARNING  TransactionRecord SpendBundle ID: b'g:\xa7\x0b5\xa8\xdb\xef\xf3Q\x84\xde4\n\xb4\r\xc8\x88D\xb7\x15\x93=~\x85\x1e6\x125tQ\xb9' not in mempool. (peer, included, error) list: [('bd27d1a278e6dfdc6794ca5319d5ec619d3310b77f972c2862de25d2a4709af0', 3, 'INVALID_FEE_TOO_CLOSE_TO_ZERO')]
2024-08-04T16:20:04.601 wallet chia.wallet.wallet         : WARNING  TransactionRecord SpendBundle ID: b'g:\xa7\x0b5\xa8\xdb\xef\xf3Q\x84\xde4\n\xb4\r\xc8\x88D\xb7\x15\x93=~\x85\x1e6\x125tQ\xb9' not in mempool. (peer, included, error) list: [('bd27d1a278e6dfdc6794ca5319d5ec619d3310b77f972c2862de25d2a4709af0', 3, 'INVALID_FEE_TOO_CLOSE_TO_ZERO')]
2024-08-04T16:20:24.549 wallet chia.wallet.wallet         : WARNING  TransactionRecord SpendBundle ID: b'g:\xa7\x0b5\xa8\xdb\xef\xf3Q\x84\xde4\n\xb4\r\xc8\x88D\xb7\x15\x93=~\x85\x1e6\x125tQ\xb9' not in mempool. (peer, included, error) list: [('bd27d1a278e6dfdc6794ca5319d5ec619d3310b77f972c2862de25d2a4709af0', 3, 'INVALID_FEE_TOO_CLOSE_TO_ZERO')]
2024-08-04T16:22:26.352 wallet chia.wallet.wallet         : WARNING  TransactionRecord SpendBundle ID: b'g:\xa7\x0b5\xa8\xdb\xef\xf3Q\x84\xde4\n\xb4\r\xc8\x88D\xb7\x15\x93=~\x85\x1e6\x125tQ\xb9' not in mempool. (peer, included, error) list: [('bd27d1a278e6dfdc6794ca5319d5ec619d3310b77f972c2862de25d2a4709af0', 3, 'INVALID_FEE_TOO_CLOSE_TO_ZERO')]
2024-08-04T16:23:24.150 wallet chia.wallet.wallet         : WARNING  TransactionRecord SpendBundle ID: b'g:\xa7\x0b5\xa8\xdb\xef\xf3Q\x84\xde4\n\xb4\r\xc8\x88D\xb7\x15\x93=~\x85\x1e6\x125tQ\xb9' not in mempool. (peer, included, error) list: [('bd27d1a278e6dfdc6794ca5319d5ec619d3310b77f972c2862de25d2a4709af0', 3, 'INVALID_FEE_TOO_CLOSE_TO_ZERO')]
2024-08-04T16:23:29.912 wallet chia.wallet.wallet         : WARNING  TransactionRecord SpendBundle ID: b'g:\xa7\x0b5\xa8\xdb\xef\xf3Q\x84\xde4\n\xb4\r\xc8\x88D\xb7\x15\x93=~\x85\x1e6\x125tQ\xb9' not in mempool. (peer, included, error) list: [('bd27d1a278e6dfdc6794ca5319d5ec619d3310b77f972c2862de25d2a4709af0', 3, 'INVALID_FEE_TOO_CLOSE_TO_ZERO')]
2024-08-04T16:24:44.844 wallet chia.wallet.wallet         : WARNING  TransactionRecord SpendBundle ID: b'g:\xa7\x0b5\xa8\xdb\xef\xf3Q\x84\xde4\n\xb4\r\xc8\x88D\xb7\x15\x93=~\x85\x1e6\x125tQ\xb9' not in mempool. (peer, included, error) list: [('bd27d1a278e6dfdc6794ca5319d5ec619d3310b77f972c2862de25d2a4709af0', 3, 'INVALID_FEE_TOO_CLOSE_TO_ZERO')]
2024-08-04T16:24:44.918 full_node chia.full_node.mempool_manager: WARNING  updating the mempool using the slow-path. peak: 01cb7c7eab3ea67b17101d91c1ae2c56ff79762ca35231321a8481e775b1c470 new-peak-prev: 3ed7bca28b7247132c0aeb28153a2aea61903a3f594f6026a1c00bd0c18c5e4a coins: not set
2024-08-04T16:24:45.644 farmer chia.pools.pool_config     : ERROR    Failed to update pool config entry for launcher_id 397837e7ad51b898e20364da60fd23aa3aa8ccff0b33f162358fdabdfc5b3c0f
2024-08-04T16:24:46.241 wallet chia.wallet.wallet         : WARNING  TransactionRecord SpendBundle ID: b'g:\xa7\x0b5\xa8\xdb\xef\xf3Q\x84\xde4\n\xb4\r\xc8\x88D\xb7\x15\x93=~\x85\x1e6\x125tQ\xb9' not in mempool. (peer, included, error) list: [('bd27d1a278e6dfdc6794ca5319d5ec619d3310b77f972c2862de25d2a4709af0', 3, 'INVALID_FEE_TOO_CLOSE_TO_ZERO')]
2024-08-04T16:24:47.055 wallet wallet_server              : WARNING  Connection failed to wss://104.248.202.108:58444/ws. Error: 403, message='Invalid response status', url=URL('wss://104.248.202.108:58444/ws')
2024-08-04T16:24:47.882 wallet chia.wallet.wallet_node    : WARNING  Peer PeerInfo(_ip=IPv4Address('207.228.217.4'), _port=58444) did not respond in time.
2024-08-04T16:24:47.882 wallet wallet_server              : WARNING  Banning 207.228.217.4 for 120 seconds
2024-08-04T16:24:48.603 wallet chia.wallet.wallet_state_manager: ERROR    Failed to add coin_state: CoinState { coin: Coin { parent_coin_info: fff7747765ab55cbecd0d84e665499ff3c65fda8c16e86960d2b2e8c14dffa7e, puzzle_hash: 3cd225c5ecb2ed63b89d2b1cbeaa91350ea69991e0fc8f4eb937182c903c335c, amount: 6968000 }, spent_height: Some(1148268), created_height: Some(1148028) }, error: list index out of range
Traceback (most recent call last):
  File "chia/wallet/wallet_state_manager.py", line 1747, in _add_coin_states
  File "chia/wallet/trade_manager.py", line 190, in coins_of_interest_farmed
IndexError: list index out of range
2024-08-04T16:24:48.724 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0x6a7d52ddf6dcdca15df7f1ddec790d48335146f47ade1044c9c8b7990e4fea2a'}
2024-08-04T16:24:48.731 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0xe0f3d67e22be8e5c58cc976ced969990d6ce522b5821e61039712ed4fefce7a5'}
2024-08-04T16:24:48.733 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0xe380747562e00a25617836ced3cc4fa986e13558a1a613c6ad9e30809a5cb99a'}
2024-08-04T16:24:48.735 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0x99e8e8e965039818545ce471780b02c1e59a8d1fd28354962a9e9009f6818326'}
2024-08-04T16:24:48.737 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0xf04dcb3d82cdc56e818e932963d7aba1a86870186ba24d89208923a3e7b23a5d'}
2024-08-04T16:24:48.739 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0x040566dfdd7e7c8fe115fabb91efc1fdd1f9e8ffe6c64dee4b72cd6787a38194'}
2024-08-04T16:24:48.741 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0xb0378028a3a1cc8e9edb3e94a4ba6dbc1efad9fc27d135b329682c9f7fb22d46'}
2024-08-04T16:24:48.743 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0xe597a8b7dcbc930c1b0ca7d31ba60b0a7af5ff7237b46bd51fb29fba7cdb0ed0'}
2024-08-04T16:24:48.746 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0xe94e7bdb57b746cd65b93f6fea5619513ff2fb36f7e997d6130375a7e363008d'}
2024-08-04T16:24:48.747 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0x4c1b3951c564a499ddbd1352b21de152c678f9e2703477d85404ddff73a32c09'}
2024-08-04T16:24:48.749 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0x7fba61a06ec30edf9751a17bd97b7d07266cd65689b0727b00f5ae85c527aad2'}
2024-08-04T16:24:48.753 wallet chia.wallet.wallet_node    : WARNING  SpendBundle has been rejected by the FullNode. {'error': 'DOUBLE_SPEND', 'status': 3, 'txid': '0x9ba71ea9e3c73c83462921612d7222ce3ec6734f99ea099061346f410fc9ae4e'}
2024-08-04T16:26:08.884 full_node full_node_server        : WARNING  Banning 183.245.13.9 for 10 seconds
2024-08-04T16:26:08.885 full_node full_node_server        : WARNING  Invalid handshake with peer PeerInfo(_ip=IPv4Address('183.245.13.9'), _port=18444). Maybe the peer is running old software.
BrandtH22 commented 1 month ago

Hey @chia-rschmidt , it looks like there might be a couple of things at play here: 1 - when making changes to a plotNFT there are two transactions that occur onchain, a transaction to leave the previous pool and a followup transaction to enter the next (replace pool with self pooling as needed). There is a built in delay between when these two transactions can occur that equates to approximately 30 minutes (can be longer based on transaction times and how full blocks are). Since testnet has nearly full blocks at all times it could well be that this specific pool change needed to take longer than 30 minutes (any delay in the first transaction being confirmed relates to a delay in the second transaction and the second might have an additional delay if blocks are full). Can you double check now that additional time has lapsed as to whether the pooling state is now correct? (if so then I imagine the above noted delays caused what you are seeing)

2 - there were a few bugs relating to pooling in rc1 that have been fixed for rc2. If it is one of these bugs (meaning 1 is not correct) then once rc2 is released please resync the wallet with the gui settings / advanced / resync option and check if the pool state is correct.

3 - if neither of the above are correct we will need to look more at the specific plotNFT and associated coinspends so let us know if the above 2 do not resolve the issues

chia-rschmidt commented 1 month ago

I was able to attempt to leave the pool after waiting another 30 minutes, and that time it worked because I left the GUI open until it left successfully. This bug was more to bring attention to the fact that closing the GUI during the leaving process caused a temporary error. If that is expected than this ticket can be closed.

BrandtH22 commented 1 month ago

Hey @chia-rschmidt , were you also working with offer files during the time that these logs were taken? (those errors appear to be offer file related)

chia-rschmidt commented 1 month ago

debug.log

Just recreated the issue on RC3. Here is the full debug log. The last thing I did before pulling this was restart the GUI to make sure it occurred.

BrandtH22 commented 1 month ago

Hey @chia-rschmidt thank you for confirming, I am able to reproduce the issue locally also. The team will review as time permits and determine the best course of action to resolve.