search

lbryio / hub

MIT License
16 stars 15 forks source link

scribe full sync failed at block 213495: db op tries to delete with incorrect existing value DELETE activated_claim_and_support #111

Closed moodyjon closed 1 year ago

moodyjon commented 1 year ago

Looks like the problems with scribe full sync are not over. ☹️ It seemed a lot faster though. Took only 1-2 hours to reach this point. 😆

I have the latest hub checked out: dcd4d7a7a8de3ab41d9fb859b910c3b962eaa2fe

python3 -m hub.scribe --db_dir ./scribe_db/ --daemon_url https:/<user>:<pass>@localhost

2022-11-11 12:23:36,039 INFO hub.service.BlockchainProcessorService:234: advanced to 213483 (took 0.021s) | +2    txs (total txs: 2060407  ) | ---3/+++3 utxos | ---0/+++0 claims | ---0/+++0 supports | abandoned: 0    /    0
2022-11-11 12:23:36,099 INFO hub.service.BlockchainProcessorService:234: advanced to 213484 (took 0.060s) | +51   txs (total txs: 2060458  ) | -101/+160 utxos | ---1/+++4 claims | ---0/+++0 supports | abandoned: 0    /    0
2022-11-11 12:23:36,122 INFO hub.service.BlockchainProcessorService:234: advanced to 213485 (took 0.023s) | +4    txs (total txs: 2060462  ) | ---5/+++7 utxos | ---0/+++0 claims | ---0/+++0 supports | abandoned: 0    /    0
2022-11-11 12:23:36,169 INFO hub.service.BlockchainProcessorService:234: advanced to 213486 (took 0.047s) | +11   txs (total txs: 2060473  ) | --17/++38 utxos | ---0/+++2 claims | ---0/+++0 supports | abandoned: 0    /    0
2022-11-11 12:23:36,194 INFO hub.service.BlockchainProcessorService:234: advanced to 213487 (took 0.025s) | +5    txs (total txs: 2060478  ) | ---6/+++9 utxos | ---0/+++0 claims | ---0/+++0 supports | abandoned: 0    /    0
2022-11-11 12:23:36,225 INFO hub.service.BlockchainProcessorService:234: advanced to 213488 (took 0.031s) | +9    txs (total txs: 2060487  ) | --13/++41 utxos | ---0/+++0 claims | ---0/+++0 supports | abandoned: 0    /    0
2022-11-11 12:23:36,248 INFO hub.service.BlockchainProcessorService:234: advanced to 213489 (took 0.024s) | +3    txs (total txs: 2060490  ) | ---4/+++5 utxos | ---0/+++0 claims | ---0/+++0 supports | abandoned: 0    /    0
2022-11-11 12:23:36,269 INFO hub.service.BlockchainProcessorService:234: advanced to 213490 (took 0.020s) | +1    txs (total txs: 2060491  ) | ---1/+++1 utxos | ---0/+++0 claims | ---0/+++0 supports | abandoned: 0    /    0
2022-11-11 12:23:36,290 INFO hub.service.BlockchainProcessorService:234: advanced to 213491 (took 0.021s) | +1    txs (total txs: 2060492  ) | ---1/+++1 utxos | ---0/+++0 claims | ---0/+++0 supports | abandoned: 0    /    0
2022-11-11 12:23:36,357 INFO hub.service.BlockchainProcessorService:234: advanced to 213492 (took 0.067s) | +67   txs (total txs: 2060559  ) | --79/+175 utxos | ---1/+++3 claims | ---0/+++0 supports | abandoned: 0    /    0
2022-11-11 12:23:36,391 INFO hub.service.BlockchainProcessorService:234: advanced to 213493 (took 0.033s) | +14   txs (total txs: 2060573  ) | --26/++73 utxos | ---0/+++1 claims | ---0/+++0 supports | abandoned: 0    /    0
2022-11-11 12:23:36,422 INFO hub.service.BlockchainProcessorService:234: advanced to 213494 (took 0.031s) | +15   txs (total txs: 2060588  ) | --15/++29 utxos | ---0/+++2 claims | ---0/+++0 supports | abandoned: 0    /    0
2022-11-11 12:23:36,442 ERROR hub.service.BlockchainProcessorService:242: advance blocks failed
Traceback (most recent call last):
  File "/Users/swdev1/hub/hub/scribe/service.py", line 229, in check_and_advance_blocks
    txo_count, time.perf_counter() - start
  File "/Users/swdev1/hub/hub/scribe/service.py", line 156, in run_in_thread_with_lock
    async def run_in_thread():
  File "/Users/swdev1/hub/hub/scribe/service.py", line 155, in run_in_thread_locked
    async def run_in_thread(self, func, *args):
  File "/opt/homebrew/Cellar/python@3.9/3.9.14/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/swdev1/hub/hub/scribe/service.py", line 1759, in advance_block
    raise ChainError(
  File "/Users/swdev1/hub/hub/db/interface.py", line 220, in unsafe_commit
    else:
  File "/Users/swdev1/hub/hub/db/interface.py", line 292, in apply_stash
    def multi_delete(self, items: typing.List[typing.Tuple[bytes, bytes]]):
  File "/Users/swdev1/hub/hub/db/revertable.py", line 175, in validate_and_apply_stashed_ops
  File "/Users/swdev1/hub/hub/db/revertable.py", line 160, in validate_and_apply_stashed_ops
    need_put.append(op)
hub.db.revertable.OpStackIntegrity: db op tries to delete with incorrect existing value DELETE activated_claim_and_support: ActivationKey(txo_type=2, tx_num=2059795, position=1) | ActivationValue(height=213469, claim_hash=8e04c56d27e6767f65a3e00158d866764972ff2d, normalized_name=holi)
vs
b"\x00\x03A\xdd\x8e\x04\xc5m'\xe6v\x7fe\xa3\xe0\x01X\xd8fvIr\xff-\x00\x03ten"
2022-11-11 12:23:36,445 ERROR hub.service.BlockchainProcessorService:2080: error while processing txs: db op tries to delete with incorrect existing value DELETE activated_claim_and_support: ActivationKey(txo_type=2, tx_num=2059795, position=1) | ActivationValue(height=213469, claim_hash=8e04c56d27e6767f65a3e00158d866764972ff2d, normalized_name=holi)
vs
b"\x00\x03A\xdd\x8e\x04\xc5m'\xe6v\x7fe\xa3\xe0\x01X\xd8fvIr\xff-\x00\x03ten"
Traceback (most recent call last):
  File "/Users/swdev1/hub/hub/scribe/service.py", line 2076, in process_blocks_and_mempool_forever
  File "/Users/swdev1/hub/hub/scribe/service.py", line 229, in check_and_advance_blocks
    txo_count, time.perf_counter() - start
  File "/Users/swdev1/hub/hub/scribe/service.py", line 156, in run_in_thread_with_lock
    async def run_in_thread():
  File "/Users/swdev1/hub/hub/scribe/service.py", line 155, in run_in_thread_locked
    async def run_in_thread(self, func, *args):
  File "/opt/homebrew/Cellar/python@3.9/3.9.14/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/swdev1/hub/hub/scribe/service.py", line 1759, in advance_block
    raise ChainError(
  File "/Users/swdev1/hub/hub/db/interface.py", line 220, in unsafe_commit
    else:
  File "/Users/swdev1/hub/hub/db/interface.py", line 292, in apply_stash
    def multi_delete(self, items: typing.List[typing.Tuple[bytes, bytes]]):
  File "/Users/swdev1/hub/hub/db/revertable.py", line 175, in validate_and_apply_stashed_ops
  File "/Users/swdev1/hub/hub/db/revertable.py", line 160, in validate_and_apply_stashed_ops
    need_put.append(op)
hub.db.revertable.OpStackIntegrity: db op tries to delete with incorrect existing value DELETE activated_claim_and_support: ActivationKey(txo_type=2, tx_num=2059795, position=1) | ActivationValue(height=213469, claim_hash=8e04c56d27e6767f65a3e00158d866764972ff2d, normalized_name=holi)
vs
b"\x00\x03A\xdd\x8e\x04\xc5m'\xe6v\x7fe\xa3\xe0\x01X\xd8fvIr\xff-\x00\x03ten"
2022-11-11 12:23:36,446 ERROR hub.service.BlockchainProcessorService:2088: fatal error in block processor loop: db op tries to delete with incorrect existing value DELETE activated_claim_and_support: ActivationKey(txo_type=2, tx_num=2059795, position=1) | ActivationValue(height=213469, claim_hash=8e04c56d27e6767f65a3e00158d866764972ff2d, normalized_name=holi)
vs
b"\x00\x03A\xdd\x8e\x04\xc5m'\xe6v\x7fe\xa3\xe0\x01X\xd8fvIr\xff-\x00\x03ten"
Traceback (most recent call last):
  File "/Users/swdev1/hub/hub/scribe/service.py", line 2081, in process_blocks_and_mempool_forever
  File "/Users/swdev1/hub/hub/scribe/service.py", line 2076, in process_blocks_and_mempool_forever
  File "/Users/swdev1/hub/hub/scribe/service.py", line 229, in check_and_advance_blocks
    txo_count, time.perf_counter() - start
  File "/Users/swdev1/hub/hub/scribe/service.py", line 156, in run_in_thread_with_lock
    async def run_in_thread():
  File "/Users/swdev1/hub/hub/scribe/service.py", line 155, in run_in_thread_locked
    async def run_in_thread(self, func, *args):
  File "/opt/homebrew/Cellar/python@3.9/3.9.14/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/swdev1/hub/hub/scribe/service.py", line 1759, in advance_block
    raise ChainError(
  File "/Users/swdev1/hub/hub/db/interface.py", line 220, in unsafe_commit
    else:
  File "/Users/swdev1/hub/hub/db/interface.py", line 292, in apply_stash
    def multi_delete(self, items: typing.List[typing.Tuple[bytes, bytes]]):
  File "/Users/swdev1/hub/hub/db/revertable.py", line 175, in validate_and_apply_stashed_ops
  File "/Users/swdev1/hub/hub/db/revertable.py", line 160, in validate_and_apply_stashed_ops
    need_put.append(op)
hub.db.revertable.OpStackIntegrity: db op tries to delete with incorrect existing value DELETE activated_claim_and_support: ActivationKey(txo_type=2, tx_num=2059795, position=1) | ActivationValue(height=213469, claim_hash=8e04c56d27e6767f65a3e00158d866764972ff2d, normalized_name=holi)
vs
b"\x00\x03A\xdd\x8e\x04\xc5m'\xe6v\x7fe\xa3\xe0\x01X\xd8fvIr\xff-\x00\x03ten"