spesmilo / electrumx

Alternative implementation of spesmilo/electrum-server
MIT License
429 stars 343 forks source link

ChainError: UTXO ... 1 not found in h table #216

Closed freerko closed 1 year ago

freerko commented 1 year ago

Hi,

I got this ChainError running ElectrumX 1.16.0 on Ubuntu 22.04 with Bitcoin Mainnet. After a restart it's the same error. Is my database corrupt, do I have to sync again or would it be possible/enough to delete the db/utxo directory?

Jun 21 03:51:20 hostname electrumx_server[7306]: INFO:ElectrumX:[11185453] SSL xx.xx.xx.xx:xx, 336 total Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[11185454] SSL xx.xx.xx.xx:xx, 335 total Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[11185455] SSL xx.xx.xx.xx:xx, 336 total Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[11185456] SSL xx.xx.xx.xx:xx, 336 total Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[11185457] SSL xx.xx.xx.xx:xx, 337 total Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:Prefetcher:cancelled; prefetcher stopping Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:SessionManager:closing down server for wss://all_interfaces:50004 Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:SessionManager:closing down server for rpc://localhost:8000 Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:SessionManager:closing down server for tcp://all_interfaces:50001 Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:SessionManager:closing down server for ssl://all_interfaces:50002 Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[11185458] SSL xx.xx.xx.xx:xx, 337 total Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[3174571] disconnected. Sent 6,366,886 bytes in 4,011 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[8482444] disconnected. Sent 2,122,360 bytes in 1,418 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[9003277] disconnected. Sent 1,856,250 bytes in 1,417 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[9492997] disconnected. Sent 1,405,474 bytes in 946 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[9565901] disconnected. Sent 1,311,673 bytes in 885 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[9632196] disconnected. Sent 1,255,573 bytes in 840 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[9744641] disconnected. Sent 1,179,884 bytes in 781 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[8878549] disconnected. Sent 1,966,940 bytes in 1,667 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[4824808] disconnected. Sent 5,140,923 bytes in 4,717 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[8162168] disconnected. Sent 2,481,437 bytes in 2,095 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[3732483] disconnected. Sent 5,820,151 bytes in 4,793 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[1584587] disconnected. Sent 8,172,797 bytes in 6,574 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[6696330] disconnected. Sent 3,379,866 bytes in 3,143 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[1597615] disconnected. Sent 7,981,682 bytes in 6,799 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[7453269] disconnected. Sent 2,939,452 bytes in 2,706 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[2085] disconnected. Sent 9,878,807 bytes in 7,502 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[7422610] disconnected. Sent 2,886,224 bytes in 2,646 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[8155382] disconnected. Sent 2,470,810 bytes in 2,073 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[3336] disconnected. Sent 9,763,534 bytes in 8,066 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[5116212] disconnected. Sent 4,773,197 bytes in 3,940 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[2413] disconnected. Sent 9,790,200 bytes in 5,819 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[4830875] disconnected. Sent 5,087,536 bytes in 4,159 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[5010553] disconnected. Sent 4,837,896 bytes in 3,954 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[2789156] disconnected. Sent 6,669,418 bytes in 4,846 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[8258712] disconnected. Sent 2,329,536 bytes in 2,134 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[9067060] disconnected. Sent 1,731,197 bytes in 1,454 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[3106617] disconnected. Sent 6,506,027 bytes in 4,948 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[9521369] disconnected. Sent 1,393,461 bytes in 1,118 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[4813] disconnected. Sent 9,731,114 bytes in 7,135 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[7093792] disconnected. Sent 3,074,959 bytes in 2,331 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[8340506] disconnected. Sent 2,231,703 bytes in 2,043 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[9755964] disconnected. Sent 1,138,637 bytes in 897 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[3099933] disconnected. Sent 6,288,134 bytes in 4,374 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[4832618] disconnected. Sent 4,919,065 bytes in 3,999 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[8186950] disconnected. Sent 2,360,717 bytes in 1,636 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[9569195] disconnected. Sent 1,292,862 bytes in 884 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[7401101] disconnected. Sent 2,901,428 bytes in 1,950 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[6529601] disconnected. Sent 3,435,383 bytes in 2,658 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[6709956] disconnected. Sent 3,152,035 bytes in 2,243 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[7093040] disconnected. Sent 3,153,235 bytes in 2,430 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[3944239] disconnected. Sent 5,448,193 bytes in 3,766 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[2432515] disconnected. Sent 7,023,239 bytes in 4,382 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[32066] disconnected. Sent 9,563,536 bytes in 6,130 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[9647767] disconnected. Sent 1,224,529 bytes in 956 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[4820456] disconnected. Sent 5,040,511 bytes in 3,637 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[6967929] disconnected. Sent 3,297,266 bytes in 2,178 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[2810] disconnected. Sent 10,095,912 bytes in 5,878 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[8763929] disconnected. Sent 2,017,738 bytes in 1,556 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[9569664] disconnected. Sent 1,320,915 bytes in 884 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[8452058] disconnected. Sent 2,188,011 bytes in 1,669 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[5684752] disconnected. Sent 3,618,940 bytes in 3,017 messages Jun 21 03:51:21 hostname electrumx_server[7306]: INFO:ElectrumX:[11132232] disconnected. Sent 11,463,430 bytes in 390 messages Jun 21 03:51:22 hostname electrumx_server[7306]: INFO:Controller:shutting down Jun 21 03:51:22 hostname electrumx_server[7306]: INFO:Controller:shutdown complete Jun 21 03:51:23 hostname electrumx_server[7306]: ERROR:electrumx:ElectrumX server terminated abnormally Jun 21 03:51:23 hostname electrumx_server[7306]: Traceback (most recent call last): Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/EGG-INFO/scripts/electrumx_server", line 35, in main Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run Jun 21 03:51:23 hostname electrumx_server[7306]: return loop.run_until_complete(main) Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete Jun 21 03:51:23 hostname electrumx_server[7306]: return future.result() Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/lib/server_base.py", line 129, in run Jun 21 03:51:23 hostname electrumx_server[7306]: await server_task Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/lib/server_base.py", line 102, in serve Jun 21 03:51:23 hostname electrumx_server[7306]: await self.serve(shutdown_event) Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/controller.py", line 131, in serve Jun 21 03:51:23 hostname electrumx_server[7306]: async with OldTaskGroup() as group: Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/aiorpcx/curio.py", line 297, in aexit Jun 21 03:51:23 hostname electrumx_server[7306]: await self.join() Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/lib/util.py", line 370, in join Jun 21 03:51:23 hostname electrumx_server[7306]: task.result() Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 700, in fetch_and_process_blocks Jun 21 03:51:23 hostname electrumx_server[7306]: async with OldTaskGroup() as group: Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/aiorpcx/curio.py", line 297, in aexit Jun 21 03:51:23 hostname electrumx_server[7306]: await self.join() Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/lib/util.py", line 370, in join Jun 21 03:51:23 hostname electrumx_server[7306]: task.result() Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 663, in _process_prefetched_blocks Jun 21 03:51:23 hostname electrumx_server[7306]: await self.check_and_advance_blocks(blocks) Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 229, in check_and_advance_blocks Jun 21 03:51:23 hostname electrumx_server[7306]: await self.run_in_thread_with_lock(self.advance_blocks, blocks) Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 212, in run_in_thread_with_lock Jun 21 03:51:23 hostname electrumx_server[7306]: return await asyncio.shield(run_in_thread_locked()) Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 211, in run_in_thread_locked Jun 21 03:51:23 hostname electrumx_server[7306]: return await run_in_thread(func, args) Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/aiorpcx/curio.py", line 57, in run_in_thread Jun 21 03:51:23 hostname electrumx_server[7306]: return await get_event_loop().run_in_executor(None, func, args) Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run Jun 21 03:51:23 hostname electrumx_server[7306]: result = self.fn(*self.args, *self.kwargs) Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 409, in advance_blocks Jun 21 03:51:23 hostname electrumx_server[7306]: undo_info = self.advance_txs(block.transactions, is_unspendable) Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 450, in advance_txs Jun 21 03:51:23 hostname electrumx_server[7306]: cache_value = spend_utxo(txin.prev_hash, txin.prev_idx) Jun 21 03:51:23 hostname electrumx_server[7306]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 646, in spend_utxo Jun 21 03:51:23 hostname electrumx_server[7306]: raise ChainError(f'UTXO {hash_to_hex_str(tx_hash)} / {tx_idx:,d} not ' Jun 21 03:51:23 hostname electrumx_server[7306]: electrumx.server.block_processor.ChainError: UTXO 288fb861ade959dcbf3be180cd081d6dd49ed41d791ea84e52fea8be446b483b / 1 not found in "h" table Jun 21 03:51:31 hostname systemd[1]: electrumx.service: Deactivated successfully. Jun 21 03:51:31 hostname systemd[1]: electrumx.service: Consumed 5d 4h 22min 54.556s CPU time. Jun 21 10:47:38 hostname electrumx_server[414]: INFO:electrumx:ElectrumX server starting Jun 21 10:47:41 hostname electrumx_server[414]: INFO:electrumx:logging level: INFO Jun 21 10:47:41 hostname electrumx_server[414]: INFO:Controller:Python version: 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0] Jun 21 10:47:41 hostname electrumx_server[414]: INFO:Controller:software version: ElectrumX 1.16.0 Jun 21 10:47:41 hostname electrumx_server[414]: INFO:Controller:aiorpcX version: 0.22.1 Jun 21 10:47:41 hostname electrumx_server[414]: INFO:Controller:supported protocol versions: 1.4-1.4.2 Jun 21 10:47:41 hostname electrumx_server[414]: INFO:Controller:event loop policy: None Jun 21 10:47:41 hostname electrumx_server[414]: INFO:Controller:reorg limit is 200 blocks Jun 21 10:47:41 hostname electrumx_server[414]: INFO:Daemon:daemon #1 at localhost:8332/ (current) Jun 21 10:47:41 hostname electrumx_server[414]: INFO:DB:switching current directory to /home/electrumx/db Jun 21 10:47:41 hostname electrumx_server[414]: INFO:DB:using leveldb for DB backend Jun 21 10:48:27 hostname electrumx_server[414]: ERROR:Daemon:starting up checking blocks. Retrying occasionally... Jun 21 10:48:43 hostname electrumx_server[414]: INFO:Daemon:running normally Jun 21 10:49:10 hostname electrumx_server[414]: INFO:DB:opened UTXO DB (for sync: True) Jun 21 10:49:10 hostname electrumx_server[414]: INFO:DB:UTXO DB version: 8 Jun 21 10:49:10 hostname electrumx_server[414]: INFO:DB:coin: BitcoinSegwit Jun 21 10:49:10 hostname electrumx_server[414]: INFO:DB:network: mainnet Jun 21 10:49:10 hostname electrumx_server[414]: INFO:DB:height: 795,271 Jun 21 10:49:10 hostname electrumx_server[414]: INFO:DB:tip: 0000000000000000000161adb0147e2712013185b8f68ec32310ae15c792701c Jun 21 10:49:10 hostname electrumx_server[414]: INFO:DB:tx count: 854,712,373 Jun 21 10:49:10 hostname electrumx_server[414]: INFO:DB:flushing DB cache at 1,200 MB Jun 21 10:49:11 hostname electrumx_server[414]: INFO:History:history DB version: 1 Jun 21 10:49:11 hostname electrumx_server[414]: INFO:History:flush count: 3,216 Jun 21 10:50:26 hostname electrumx_server[414]: INFO:DB:deleted 3,016 stale undo entries Jun 21 10:50:27 hostname electrumx_server[414]: INFO:SessionManager:RPC server listening on localhost:8000 Jun 21 10:50:28 hostname electrumx_server[414]: INFO:Prefetcher:catching up to daemon height 795,302 (31 blocks behind) Jun 21 10:56:49 hostname electrumx_server[414]: INFO:Prefetcher:cancelled; prefetcher stopping Jun 21 10:56:49 hostname electrumx_server[414]: INFO:SessionManager:closing down server for rpc://localhost:8000 Jun 21 10:56:49 hostname electrumx_server[414]: INFO:Controller:shutting down Jun 21 10:56:49 hostname electrumx_server[414]: INFO:Controller:shutdown complete Jun 21 10:56:49 hostname electrumx_server[414]: ERROR:electrumx:ElectrumX server terminated abnormally Jun 21 10:56:49 hostname electrumx_server[414]: Traceback (most recent call last): Jun 21 10:56:49 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/EGG-INFO/scripts/electrumx_server", line 35, in main Jun 21 10:56:49 hostname electrumx_server[414]: File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run Jun 21 10:56:49 hostname electrumx_server[414]: return loop.run_until_complete(main) Jun 21 10:56:49 hostname electrumx_server[414]: File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete Jun 21 10:56:50 hostname electrumx_server[414]: return future.result() Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/lib/server_base.py", line 129, in run Jun 21 10:56:50 hostname electrumx_server[414]: await server_task Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/lib/server_base.py", line 102, in serve Jun 21 10:56:50 hostname electrumx_server[414]: await self.serve(shutdown_event) Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/controller.py", line 131, in serve Jun 21 10:56:50 hostname electrumx_server[414]: async with OldTaskGroup() as group: Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/aiorpcx/curio.py", line 297, in aexit Jun 21 10:56:50 hostname electrumx_server[414]: await self.join() Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/lib/util.py", line 370, in join Jun 21 10:56:50 hostname electrumx_server[414]: task.result() Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 700, in fetch_and_process_blocks Jun 21 10:56:50 hostname electrumx_server[414]: async with OldTaskGroup() as group: Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/aiorpcx/curio.py", line 297, in aexit Jun 21 10:56:50 hostname electrumx_server[414]: await self.join() Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/lib/util.py", line 370, in join Jun 21 10:56:50 hostname electrumx_server[414]: task.result() Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 663, in _process_prefetched_blocks Jun 21 10:56:50 hostname electrumx_server[414]: await self.check_and_advance_blocks(blocks) Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 229, in check_and_advance_blocks Jun 21 10:56:50 hostname electrumx_server[414]: await self.run_in_thread_with_lock(self.advance_blocks, blocks) Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 212, in run_in_thread_with_lock Jun 21 10:56:50 hostname electrumx_server[414]: return await asyncio.shield(run_in_thread_locked()) Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 211, in run_in_thread_locked Jun 21 10:56:50 hostname electrumx_server[414]: return await run_in_thread(func, args) Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/aiorpcx/curio.py", line 57, in run_in_thread Jun 21 10:56:50 hostname electrumx_server[414]: return await get_event_loop().run_in_executor(None, func, args) Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run Jun 21 10:56:50 hostname electrumx_server[414]: result = self.fn(self.args, **self.kwargs) Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 409, in advance_blocks Jun 21 10:56:50 hostname electrumx_server[414]: undo_info = self.advance_txs(block.transactions, is_unspendable) Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 450, in advance_txs Jun 21 10:56:50 hostname electrumx_server[414]: cache_value = spend_utxo(txin.prev_hash, txin.prev_idx) Jun 21 10:56:50 hostname electrumx_server[414]: File "/usr/local/lib/python3.10/dist-packages/e_x-1.16.0-py3.10.egg/electrumx/server/block_processor.py", line 646, in spend_utxo Jun 21 10:56:50 hostname electrumx_server[414]: raise ChainError(f'UTXO {hash_to_hex_str(tx_hash)} / {tx_idx:,d} not ' Jun 21 10:56:50 hostname electrumx_server[414]: electrumx.server.block_processor.ChainError: UTXO 288fb861ade959dcbf3be180cd081d6dd49ed41d791ea84e52fea8be446b483b / 1 not found in "h" table Jun 21 10:56:50 hostname systemd[1]: electrumx.service: Deactivated successfully. Jun 21 10:56:50 hostname systemd[1]: electrumx.service: Consumed 20.224s CPU time. root@hostname:~#

SomberNight commented 1 year ago

Yes, I think the db got corrupted :/ You have to resync electrumx from genesis. If it happens again, it could be that the corruption originates from bitcoind's db; or that you have failing hardware (e.g. memory or disk).

freerko commented 1 year ago

Thanks, will do!