search

lightningnetwork / lnd

Lightning Network Daemon ⚡️
MIT License
7.71k stars 2.09k forks source link

Rescan is going very slowly after --reset-wallet-transactions #6581

Open Overtorment opened 2 years ago

Overtorment commented 2 years ago

Background

After having issues with UTXO set I decided to try --reset-wallet-transactions. All is working as expected except its going extremely slow. There is no indication in logs what block lnd is rescaning, but i can get a hint of it withlistunspent and peeping a specific txid and it's confirmations, and judging by it lnd is still working on a date ‎2020-05-05 02:09 (barely advanced, wallet birthday is somewhere in 2020-04-xx).

Looking at stats, lnd is eating 50% cpu on each of 4 cores on the server. Disk io is not utilized.

I sense the issue might be that this node has 300k onchain addresses. Is there something we can do?

PS. v0.15.0-beta.rc3

C-Otto commented 2 years ago

I think the output is shown every 50k blocks or so, which might take a while. On my node it took ~8 hours (birthday in 2018 or so).

Overtorment commented 2 years ago

its already taking a week

guggero commented 2 years ago

I don't think there's anything you can do now that the rescan is already running. It also shouldn't be aborted, otherwise some addresses might not be recognized properly later on. My assumption is that going through each transaction in each block and looking at every output and comparing every output to the 300k onchain addresses is just a lot of work (lots of nested loops with the inner one being very large with 300k iterations).

This should probably be optimized in the btcwallet code. But 300k addresses is also just a lot...