search

tari-project / tari

The Tari protocol
https://tari.com
BSD 3-Clause "New" or "Revised" License
354 stars 219 forks source link

Wallet recovery intermittent error `UNIQUE constraint failed: scanned_blocks.header_hash` (commit: 44e3731) #5912

Open hansieodendaal opened 1 year ago

hansieodendaal commented 1 year ago

Describe the bug Wallet recovery has an intermittent UNIQUE constraint failed: scanned_blocks.header_hash error - when connecting to the same node from a clean slate this error sometimes occurs other times not.

Error in the log:

2023-11-07 08:02:06.777449900 [wallet::utxo_scanning] WARN  Failed to scan UTXO's from base node 7dcc333714f9b31ebda93edec1: Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``
2023-11-07 08:02:06.777948000 [wallet::recovery] WARN  Attempt 1/100: Failed to complete wallet recovery Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``.

Console output: 1st attempt exhausted all base nodes

minotari_console_wallet.exe --base-path . --network esmeralda --config %cd%\config\config.toml --log-config %cd%\config\log4rs_console_wallet.yml --recover --seed-words "cage churn upper theory primary expand sail fan cart ghost mobile example mammal icon detect flavor inspire nest number enemy science write great level"
Initializing logging according to "node_test\\config\\log4rs_console_wallet.yml"
Minotari Console Wallet running... (Recovery mode started)
Starting recovery...

Press Ctrl-C to stop the recovery process

Connecting to base node 7dcc333714f9b31ebda93edec1...
OK (latency = 576.70µs)
Attempt 1/100: Failed to complete wallet recovery Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``.
Connecting to base node f750562a18979c2c2b85ec13db...
OK (latency = 7.45ms)
Attempt 1/100: Failed to complete wallet recovery Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``.
Connecting to base node 57f8de99f1371a7e1f5d0110f2...
OK (latency = 366.30µs)
Attempt 1/100: Failed to complete wallet recovery Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``.
Connecting to base node 4aad2340c707b45d73a7b8ac8d...
OK (latency = 370.20µs)
Attempt 1/100: Failed to complete wallet recovery Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``.
Connecting to base node 27491f43dbf7043d25441097f8...
OK (latency = 451.80µs)
Attempt 1/100: Failed to complete wallet recovery Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``.
Connecting to base node 4e2266d72446607dc45567871c...
OK (latency = 1.84s)
Attempt 1/100: Failed to complete wallet recovery Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``.
Connecting to base node 3ac3706045087831ae7b3b92e1...
Base node connection error to 3ac3706045087831ae7b3b92e1 (retries 1 of 100: ConnectionFailed: Failed to connect to peer within the maximum number of attempts)
Attempt 1/100: Failed to complete wallet recovery Connectivity error: `ConnectionFailed: Failed to connect to peer within the maximum number of attempts`.
Connecting to base node c73c8e0c4043dfda4bb85d504a...
Base node connection error to c73c8e0c4043dfda4bb85d504a (retries 1 of 100: ConnectionFailed: Failed to connect to peer within the maximum number of attempts)
Attempt 1/100: Failed to complete wallet recovery Connectivity error: `ConnectionFailed: Failed to connect to peer within the maximum number of attempts`.
Connecting to base node fffffffb1801b9dd652dc74b19...
Base node connection error to fffffffb1801b9dd652dc74b19 (retries 1 of 100: ConnectionFailed: Failed to connect to peer within the maximum number of attempts)
Attempt 1/100: Failed to complete wallet recovery Connectivity error: `ConnectionFailed: Failed to connect to peer within the maximum number of attempts`.
Connecting to base node 111111df4003f0c2c827eddad8...
Base node connection error to 111111df4003f0c2c827eddad8 (retries 1 of 100: ConnectionFailed: Failed to connect to peer within the maximum number of attempts)
Attempt 1/100: Failed to complete wallet recovery Connectivity error: `ConnectionFailed: Failed to connect to peer within the maximum number of attempts`.
Attempt 1/100: Failed to complete wallet recovery No new peers to try after this round.
Connecting to base node 7dcc333714f9b31ebda93edec1...
OK (latency = 434.20µs)
^

2nd attempt (clean slate) successfully started recovery from a base node it had an issue with on the 1st attempt

minotari_console_wallet.exe --base-path . --network esmeralda --config %cd%\config\config.toml --log-config %cd%\config\log4rs_console_wallet.yml --recover --seed-words "cage churn upper theory primary expand sail fan cart ghost mobile example mammal icon detect flavor inspire nest number enemy science write great level"
Initializing logging according to "node_test\\config\\log4rs_console_wallet.yml"
Minotari Console Wallet running... (Recovery mode started)
Starting recovery...

Press Ctrl-C to stop the recovery process

Connecting to base node 7dcc333714f9b31ebda93edec1...
OK (latency = 284.70µs)
Attempt 1/100: Failed to complete wallet recovery Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``.
Connecting to base node f750562a18979c2c2b85ec13db...
OK (latency = 291.70µs)
Attempt 1/100: Failed to complete wallet recovery Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``.
Connecting to base node 57f8de99f1371a7e1f5d0110f2...
OK (latency = 381.30µs)
Attempt 1/100: Failed to complete wallet recovery Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``.
Connecting to base node 4aad2340c707b45d73a7b8ac8d...
OK (latency = 371.70µs)
Attempt 1/100: Failed to complete wallet recovery Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``.
Connecting to base node 27491f43dbf7043d25441097f8...
OK (latency = 543.40µs)
Attempt 1/100: Failed to complete wallet recovery Wallet storage error: `Diesel error: `UNIQUE constraint failed: scanned_blocks.header_hash``.
Connecting to base node 4e2266d72446607dc45567871c...
OK (latency = 1.87s)
2023-11-07 08:08:39.017099700 +02:00: Recovery process 7% complete (Block 200 of 2686).
2023-11-07 08:08:39.039035400 +02:00: Recovery process 11% complete (Block 300 of 2686).
2023-11-07 08:08:39.064433800 +02:00: Recovery process 14% complete (Block 400 of 2686).

To Reproduce Perform wallet recovery for multiple wallets

SWvheerden commented 10 months ago

We need more info here, struggling to replicate this.

Looking at the code, this is either the base node supplying a duplicate header, or the wallet is requesting a header it already has.