search

decred / dcrdex

The Decred Decentralized Exchange (DEX), powered by atomic-swaps.
Other
183 stars 90 forks source link

BTC Wallet Sync Issues #2899

Open martonp opened 1 month ago

martonp commented 1 month ago

There are multiple issues when syncing a BTC SPV wallet, especially one that was initialized with an old seed where the birthday will be Jun 2021.

  1. During the initial sync (the wallet was never restarted), the CFilters will be fetched one at a time. I have seen each CFilter taking 0.5 - 1 second to arrive. This call is happening over here: https://github.com/lightninglabs/neutrino/blob/a50a92ab8271ea441ef3894fb669ca95be7fa40d/rescan.go#L960. This process should be done in larger batches and parallelized.
  2. If the wallet is restarted, the CFilters will be fetched in batches of 1000, but this is still extremely slow. Each batch of 1000 takes between 1.5 mins - 3.5 mins to arrive. There is a TODO in btcwallet about parallelizing this process that would probably fix this: https://github.com/btcsuite/btcwallet/blob/db3a4a2543bdc7ebc1e992b2f7fd01e4abd257a3/wallet/wallet.go#L834
  3. When the wallet is restarted, it cannot be shut down until the sync has completed. This issue does not happen if this is the initial sync in which the wallet was never restarted. The reason for this is that the long running process in which Cfilters are requested 1000 at a time is unmanaged. A context needs to be passed into recoverScopedAddresses (see link in wallet.go above) that kills this.
buck54321 commented 1 month ago

https://github.com/btcsuite/btcwallet/pull/944 fixes 3

matthawkins90 commented 1 month ago

Wow, nice job finding these issues, as well as @buck54321 already fixing 3.