Closed eugenesan closed 1 month ago
It would useful to be able to load multiple sparrow wallets in parallel.
My suggestion would be to use a Electrum server like Fulcrum for this use case. If you are opening and closing many wallets (new and old), this is likely to be a much better experience.
If it's not possible to add new addresses/xpubs while single cormorant wallet is syncing maybe we should consider using multiple Core wallets for each Sparrow wallet since Bitcoin Core is capable of doing so with built in wallets.
Unfortunately, this wouldn't help - see https://github.com/bitcoin/bitcoin/issues/11756
Test 1
Unfortunately there is no way for Sparrow to tell AFAIK if Core has been restarted during a wallet scan. The likely RPC call getwalletinfo
does not return any information to indicate this - after the restart it simply indicates "scanning": false
for the wallet. Rescans can be triggered in Sparrow by setting the birthday of the wallet to a little earlier, see Settings tab > Advanced.
Test 2
I'll need more specific instructions to recreate. FWIW I switch wallets between Core and Electrum frequently when testing, and I haven't observed this.
Test 3
Again, more specific information is needed to recreate.
Is there a way to stop current sync when a new one is required?
Not currently, although Core does offer the abortrescan
RPC call. Implementing this in Sparrow might cause more issues than it solves though.
With pruned nodes, do we need to run Sparrow periodically, before Core prunes the blocks Sparrow missed?
No - so long as Core is kept running no transactions will be missed.
@craigraw Thank you for your detailed response.
I am not trying to frequently load multiple different wallets. All I need is a reliable procedure to switch to Cormorant, "sync-up" a few wallets and optionally perform Core pruning.
I understand that Full-Node/Fulcrum is the proper solution and that's what I am using ATM. But since not everyone can afford Full-Node and Sparrow documentation mentions support of pruned nodes with Cormorant, some users (including me) might want to use it.
I tried a few times and every time resulting Cormorant wallet in Core contained different set of addresses and descriptors. Seems like when multiple wallets are loaded, Cormorant mis-configures some descriptors or Core can't handle changes to cormorant wallet properly once rescanblockchain
is ongoing.
For example when querying several synced cormorant wallets using listreceivedbyaddress
and listdescriptors
:
"active": false
and sometimes "active": true
"internal": false
listreceivedbyaddress
and as descriptors "desc": "addr()
"next": 0, "next_index": 0
and once with "next": 1, "next_index": 1
Also, all wallets were missing some transactions when displayed in the core UI even after manual rescanblockchain
.Something must be done about this issue or at least there should be a warning regarding multi-wallet use with Cormorant.
I believe you misinterpret the output you are seeing. For example:
Sometimes descriptors for the same xpub contain "active": false and sometimes "active": true Sometimes some descriptors contain "internal": false
Both of these flags are used by the Core wallet internally, and have no real impact on Cormorant. See below for why the "same xpub" appears.
Every test cormorant wallet had different number of addresses.
It's hard to comment on this without specific instructions to recreate it.
Each xpub descriptor appears twice, once with "next": 0, "next_index": 0 and once with "next": 1, "next_index": 1
Because multipath descriptor support is not yet merged, Cormorant imports 2 descriptors for each wallet - the receive chain, and the change chain. They look similar, but they are not the same.
Something must be done about this issue or at least there should be a warning regarding multi-wallet use with Cormorant.
You have yet to demonstrate any issue. The way to go about it is this - from a clean start (ie an empty cormorant
wallet) perform and document a specific set of steps. Document the output and note any issues. Then start over and repeat the process, following exactly the same set of steps. Make sure you recreate the exact same output, or document how the output has changed. Use testnet to avoid using real funds, and share all the steps needed to recreate here.
Once again, thank you for detailed response.
I followed your advice and created 2 wallets on testnet4.
Here are the steps I took:
Result:
listdescriptors
listreceivedbyaddress
. Both addresses are from wallet1What's missing is the address in cormorant wallet that received transaction in step 9. Core also doesn't show the transaction from step 9. While sparrow shows everything as expected, I am concerned that if I would recreate from backup the 2 wallets in Sparrow, it wouldn't fetch all transactions using Cormorant library, especially if Core is pruned.
I guess the Core is not deriving the address despite having the corresponding XPUB descriptor. Looking at the wallet that Core created, XPUB/XPRIV descriptors contain derivation path while cormorant wallet does not. Maybe that is the problem?
What's missing is the address in cormorant wallet that received transaction in step 9.
This is because the address is derived from the descriptor.
Core also doesn't show the transaction from step 9.
The Core cormorant
wallet is part of the internal implementation of the Cormorant library - it is not intended to be used outside of Sparrow, and neither is such use supported.
While sparrow shows everything as expected, I am concerned that if I would recreate from backup the 2 wallets in Sparrow, it wouldn't fetch all transactions using Cormorant library, especially if Core is pruned.
So long as you have the seed, you can create the descriptor. So long as you have the descriptor, you can find all the transactions, in Sparrow or any other standards-compliant wallet. The internal implementation details of Cormorant aren't relevant to this process.
It appears we have moved some way from the original issue as titled here. Can we close this off?
Thank you for explaining.
I don't think we moved away from the original issue. Leaving aside all the technical details (which I admittedly might have got wrong) , I am still not clear on how to properly sync multiple Sparrow wallets with Cormorant to avoid "incomplete sync" issues?
Also, with pruned Core + Cormorant + multiple Sparrow wallets, will I have to re-download the whole Core DB if one of the Sparrow wallet files is lost?
Leaving aside all the technical details (which I admittedly might have got wrong) , I am still not clear on how to properly sync multiple Sparrow wallets with Cormorant to avoid "incomplete sync" issues?
Best advice is to avoid stopping Core while it is scanning, which creates an incomplete scan Sparrow is not able to detect as it is not reported by the Core API.
Also, with pruned Core + Cormorant + multiple Sparrow wallets, will I have to re-download the whole Core DB if one of the Sparrow wallet files is lost?
No, the Sparrow wallet files aren't important with pruning - it's keeping the Core cormorant
wallet open while Core is receiving new blocks. See here for more info: https://bitcoin.stackexchange.com/questions/88382/background-information-on-warning-when-closing-wallet-in-pruned-mode/88386#88386
Thank you very much.
One more question and we can close the issue.
What do you think about the following procedure to avoid multiple re-scans with multiple wallets:
Yes, that should work fine.
I might be just confused by how wallets work with Cormorant so please bear with me.
I've noticed that Cormorant waits for previous wallet sync to complete before start syncing newly loaded Sparrow wallets. (At least that's how it looks like visually and in the logs).
It would useful to be able to load multiple sparrow wallets in parallel.
If it's not possible to add new addresses/xpubs while single cormorant wallet is syncing maybe we should consider using multiple Core wallets for each Sparrow wallet since Bitcoin Core is capable of doing so with built in wallets.
In addition there seems be a problem with how Cormorant handles multiple wallets:
Test 1
Test 2
Test 3
Wallet refresh fixes the issue but requires another full sync. Is there a way to stop current sync when a new one is required?
P.S. Not directly related but; With pruned nodes, do we need to run Sparrow periodically, before Core prunes the blocks Sparrow missed? Or, Cormorant wallet in Core contains all the xpubs and Core will save new transaction for Sparrow fetch later?