Wallet account recovery from HD seed (BIP44)

[braydonf]

Versions bcoin v1.0.2

Expected

BIP44 HD wallets, including watch-only, to follow the specification that:

Software should prevent a creation of an account if a previous account does not have a transaction history (meaning none of its addresses have been used before). https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki#account

For the purposes of wallet recovery from the master key (mnemonic phrase):

Software needs to discover all used accounts after importing the seed from an external source. Such an algorithm is described in "Account discovery" chapter. https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki#account

Actual

Accounts can be created with the previous accounts not having a transaction history. This creates an issue with wallet recovery from the seed. In this case the total number of accounts is necessary to be known to fully recover, that is potentially 2 ^ 31 - 1 possible accounts that need 20 keys to be derived and scanned (~43 billion addresses). The problem would be worse with a non-incremental account index.

Reproduce

Start bcoin:

./bin/bcoin --network=regtest

Create wallet (using bclient):

./bin/bwallet-cli --network=regtest mkwallet --id=alice

View accounts:

./bin/bwallet-cli --network=regtest --id=alice account list

One "default" account will be listed.

Create new accounts:

./bin/bwallet-cli --network=regtest --id=alice account create purple

List all of the accounts:

./bin/bwallet-cli --network=regtest --id=alice account list
[
  "default",
  "blue",
  "yellow",
  "green",
  "orange",
  "purple"
]

Reproduce (watch-only)

Start bcoin:

./bin/bcoin --network=regtest

Create watch-only wallet with m'/44'/0'/0' (using bclient):

./bin/bwallet-cli --network=regtest mkwallet --id=bob --watch-only=true --account-key=tpubDCBWBScQPGv4Xk3JSbhw6wYYpayMjb2eAYyArpbSqQTbLDpphHGAetB6VQgVeftLML8vDSUEWcC2xDi3qJJ3YCDChJDvqVzpgoYSuT52MhJ

This account has zero transactions.

View accounts:

./bin/bwallet-cli --network=regtest --id=bob account list

One "default" account will be listed.

Create a new account for the wallet with m'/44'/0'/1':

./bin/bwallet-cli --network=regtest --id=bob account create purple --account-key=tpubDCBWBScQPGv4a6Co16myUDzcN7Uxjc9KgrvfeANX5ZkoPrjbyzj2WbY7Frx99wT4zGLCobX4TEjv8qL3mvJ3uKoHZiKqkgKWN6rcK3NAdLv

Create another account for the wallet with m'/44'/0'/2':

./bin/bwallet-cli --network=regtest --id=bob account create green --account-key=tpubDCBWBScQPGv4cFvS6Jkc4M6HBu16uHiTNyZHmZqsZEYMh4nP9oofzxmwcDADyyhkLMXUwwcypHNFntKXjc8tMkMgWnfmTgkyHR4hjNWLBWM

List all the accounts:

./bin/bwallet-cli --network=regtest --id=bob account list
[
  "default",
  "purple",
  "green"
]

[braydonf]

I've drafted a BIP https://gist.github.com/braydonf/1b55622fd4a3db682cf322019393eb95 for P2WSH multi-account multi-signature wallets (as relevant to bcoin) that can go along with BIP44, BIP49 and BIP84.

It describes account import and address discovery for the multisig case: https://gist.github.com/braydonf/1b55622fd4a3db682cf322019393eb95#shared-account-import

[braydonf]

An additional proposal specific to bcoin wallet stategies for key management: https://gist.github.com/braydonf/79b972eb0be48e9cfe11dc2a6b46699b

[pinheadmz]

Just adding to this discussion: We have an API call backup http://bcoin.io/api-docs/?shell--cli#wallet-backup that makes a copy of the walletDB's levelDB directory to <path/wallet>. (The directory name must be wallet)

Then I was able to launch bcoin --no-wallet and then bwallet --prefix=<directory-that-encloses-path/wallet>

...and observed that my xpub-imported watch-only wallets loaded in sync with all history and balance right away.

[IMPranshu]

I recently did a clean installation of Ubuntu and formatted my drive, hence lost access to my previous bcoin wallet and its funds. But I do have the wallet's seed phrase with me and looking to recover the wallet with the coins.

[IMPranshu]

Found this link somewhat helpful in this scenario. Here we can provide the HD key i.e. seed phrase while creating the wallet and hence can gain back the access to the wallet I guess. https://bcoin.io/api-docs/?shell--cli#create-a-wallet

I took the following steps bwallet-cli mkwallet $id master=$oldseedpharse this made a new wallet I guess and the coins were not showing up. Then I saw that I have to reset the block header for "SPV" node so ran this command, bcoin-cli reset 1000 But I got the following error

    at Timeout.timeout.setTimeout (/home/pk/corenode/bcoin/node_modules/brq/lib/request.js:376:19)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)

then again running the same command I got the following msg Chain has been reset.

Then stopping and re-running bcoin --spv The chain is getting resynced. Let's see what will be the coin balance after that.

[IMPranshu]

I think this cmd bwallet-cli mkwallet $id master=$oldseedpharse didn't work as I am seeing a new seed phrase for the above-made wallet.

[pinheadmz]

@IMPranshu reset will usually timeout. That's because the rpc command doesnt return a success until the entire chain has been reset, which takes a lot longer than the timeout (of ~15 seconds?) It's the same problem with full node rescan and other API calls that run background processes.

I think this cmd bwallet-cli mkwallet $id master=$oldseedpharse didn't work as I am seeing a new seed phrase for the above-made wallet.

this is incorrect syntax.

correct:

bwallet-cli mkwallet --id=whatever --mnemonic=<word... word... word...>

[pinheadmz]

https://bcoin.io/api-docs/#create-a-wallet

[IMPranshu]

https://bcoin.io/api-docs/#create-a-wallet

Yes, I was able to create the wallet using the command but the transactions are not showing up in the wallet info.

[IMPranshu]

@IMPranshu reset will usually timeout. That's because the rpc command doesnt return a success until the entire chain has been reset, which takes a lot longer than the timeout (of ~15 seconds?) It's the same problem with full node rescan and other API calls that run background processes.

I remember in the regtest the rescan cmd does the job somehow. But here as you said and I checked rescan isn't working.

What should I do in order to get the old transaction to show up in the wallet?

Currently, I am running Bcoin in spv mode

I noticed that when I tried to run the reset command it showed an error

pk@pk-Bitcoinmaxi:~/corenode/bcoin$ bcoin-cli reset 1000
Error: Request timed out.
    at Timeout.timeout.setTimeout (/home/pk/corenode/bcoin/node_modules/brq/lib/request.js:376:19)
    at ontimeout (timers.js:436:11)
    at tryOnTimeout (timers.js:300:5)
    at listOnTimeout (timers.js:263:5)
    at Timer.processTimers (timers.js:223:10)

But when I restarted the bcoin node it automatically started to scan the blockchain from the top.

[IMPranshu]

Versions bcoin v1.0.2

Expected

BIP44 HD wallets, including watch-only, to follow the specification that:

Software should prevent a creation of an account if a previous account does not have a transaction history (meaning none of its addresses have been used before). https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki#account

I think the above scenario is mentioned In the docs https://github.com/bcoin-org/bcoin/blob/master/docs/wallet-system.md#deviation-from-bip44

Unlike BIP44, bcoin does not limit account depth and a new account can be created after an empty account. This can create issues with deterministic account discovery from the master node (seed) as there are 2 ^ 31 - 1 (worst case) possible accounts to search.

Is this the same @braydonf or am I somewhere wrong?