BIP 32/39/44 Interoperability - wallet recovery between implementations?

skaht commented 7 years ago

Ran across this article - Why do my BIP32 wallets disagree? Thought I would try the m/44'/60'/0'/0/0 test vector on bx to see the results. Using Bitcoin's version of 76066276 is acceptable because the private keys are elliptic curve keys, not extended keys. Ethereum uses the last 20 bytes of a 256-bit Keccak hash of a secp256k1 generated public address as the Ethereum address.

_Uncompressed private key and Ethereum public address calculations:_

% echo b96e9ccb774cc33213cbcb2c69d3cdae17b0fe4888a1ccd343cbd1a17fd98b18 | bx ec-to-public -u 0405b7d0996e99c4a49e6c3b83288f4740d53662839eab1d97d14660696944b8bbe24fabdd03888410ace3fa4c5a809e398f036f7b99d04f82a012dca95701d103

The leading 0x04 must be dropped before taking the Keccak hash to determine the Ethereum address.

% ./keccak -256 05b7d0996e99c4a49e6c3b83288f4740d53662839eab1d97d14660696944b8bbe24fabdd03888410ace3fa4c5a809e398f036f7b99d04f82a012dca95701d103 0AB3387A148B3C4B18C333FCAC39B311DCEB2A4B2F5D8461C1CDAF756F4F7AE9

The bolded 20 byte Ethereum address immediately above matches the "Otherwise" result in the article up top :-)

For grins, I used the Trezor recovery instructions for the 12 word recovery seed ("radar blur cabbage chef fix engine embark joy scheme fiction master release") to see what the results are for the extended M/44'/0'/0'/0 xpub key to contrast to bx calculations. The resulting xpub key resulting from two repeated Trezor restore sessions without using a password or PIN was:

xpub6DHi64TFkDPx2AH4q2ku3vX9LJYNpTis5tLrET8Sb9irp174eCkgtAnvBpyzQXgrtmF31Lrq4gTMGFUGcjJicMu9LdueVdqt6FZ2Wzcg8Fj

% echo "radar blur cabbage chef fix engine embark joy scheme fiction master release" | bx mnemonic-to-seed | bx hd-new -v 76066276 | bx hd-private -d -i 44 | bx hd-private -d -i 0 | bx hd-public -d -i 0 xpub6DHi64TFkDPx2AH4q2ku3vX9LJYNpTis5tLrET8Sb9irp174eCkgtAnvBpyzQXgrtmF31Lrq4gTMGFUGcjJicMu9LdueVdqt6FZ2Wzcg8Fj

Deprecates this older posting that went below the BIP 44 account level to the change level:

% echo "radar blur cabbage chef fix engine embark joy scheme fiction master release" | bx mnemonic-to-seed | bx hd-new -v 76066276 | bx hd-private -d -i 44 | bx hd-private -d -i 0 | bx hd-private -d -i 0 | bx hd-public -i 0 xpub6FC7cZyPDk6itiQCbXQaV6Lt7x3d4MnokUF1kPGsgcQaBd72Ee6DqweH146cB9xekp7HBnyCdaGDFzcbASYd5vU1kGkayv5oj9BZKubAasr

evoskuil commented 7 years ago

As you know we use the test vectors from both BIP39 and BIP32, so it's a bit curious for sure. I'm pretty sure that we aren't dropping any leading zeroes :).

Also, we do return an error when a valid private key is not produced. For example:

    ec_secret secret(new_key(seed));

    if (secret == null_hash)
    {
        error << BX_EC_NEW_INVALID_KEY << std::endl;
        return console_result::failure;
    }

skaht commented 7 years ago

It appears that Trezor has the key recovery interoperability issue, and not the libbitcoin bx HD wallet.
Using bx, I was able to reproduce 0xac39b311dceb2a4b2f5d8461c1cdaf756f4f7ae9 Ethereum address. I forgot to drop the leading 0x04 from the uncompressed public key before taking the keccak hash. Used https://www.myetherwallet.com/ JSON outputs to ascertain that I forgot to drop the leading 0x04. Will close this issue in a few days.

evoskuil commented 7 years ago

Great, thanks for the update. It seemed from the article that he thought the Trezor implementation was the not broken one. It would be great if you track run that one down as well!

skaht commented 7 years ago

We will see what comes from this posting https://www.reddit.com/r/TREZOR/comments/52zeew/bip_323944_seed_portability/...

evoskuil commented 7 years ago

Looks like BX and Trezor are both correct and it was an issue of an incorrect path having been documented (by Trezor?).

skaht commented 7 years ago

I concur. This is how to apply bx to compute the extended public key for account number 0 to match the extended key Trezor makes available within https://wallet.mytrezor.com/#/.

% echo "radar blur cabbage chef fix engine embark joy scheme fiction master release" | bx mnemonic-to-seed | bx hd-new -v 76066276 | bx hd-private -d -i 44 | bx hd-private -d -i 0 | bx hd-public -d -i 0 xpub6DHi64TFkDPx2AH4q2ku3vX9LJYNpTis5tLrET8Sb9irp174eCkgtAnvBpyzQXgrtmF31Lrq4gTMGFUGcjJicMu9LdueVdqt6FZ2Wzcg8Fj

The first four addresses below match results from Trezor.

% echo xpub6DHi64TFkDPx2AH4q2ku3vX9LJYNpTis5tLrET8Sb9irp174eCkgtAnvBpyzQXgrtmF31Lrq4gTMGFUGcjJicMu9LdueVdqt6FZ2Wzcg8Fj | bx hd-public -i 0 | bx hd-public -i 0 | bx hd-to-ec | bx ec-to-address -v 0 1NAW6zzKT5zjtd73nVP86mtv1etp7GfThv

% echo xpub6DHi64TFkDPx2AH4q2ku3vX9LJYNpTis5tLrET8Sb9irp174eCkgtAnvBpyzQXgrtmF31Lrq4gTMGFUGcjJicMu9LdueVdqt6FZ2Wzcg8Fj | bx hd-public -i 0 | bx hd-public -i 1 | bx hd-to-ec | bx ec-to-address -v 0 19bmBLXGnxKmoE4KPmft5aE9o82rvtXnZV

% echo xpub6DHi64TFkDPx2AH4q2ku3vX9LJYNpTis5tLrET8Sb9irp174eCkgtAnvBpyzQXgrtmF31Lrq4gTMGFUGcjJicMu9LdueVdqt6FZ2Wzcg8Fj | bx hd-public -i 0 | bx hd-public -i 2 | bx hd-to-ec | bx ec-to-address -v 0 1kFFjCjBiF13dNzVnSA4mQzhDN5r8XHqH

% echo xpub6DHi64TFkDPx2AH4q2ku3vX9LJYNpTis5tLrET8Sb9irp174eCkgtAnvBpyzQXgrtmF31Lrq4gTMGFUGcjJicMu9LdueVdqt6FZ2Wzcg8Fj | bx hd-public -i 0 | bx hd-public -i 3 | bx hd-to-ec | bx ec-to-address -v 0 1DP9wsHuMDqCDZP1oYM5waov3cvK4nwL2q

evoskuil commented 7 years ago

Cool, thanks for posting. I consider BX a very solid production quality tool and am always interested in discovering potential issues. AFAIC it's a standard against which other implementations can be measured :).

libbitcoin / libbitcoin-explorer

BIP 32/39/44 Interoperability - wallet recovery between implementations? #380