BIP-32 (HD) key encodings insufficient for altchains.

[evoskuil]

HD keys have a network prefix that is distinct for public and private keys (for bitcoin mainnet there is xpub / xprv). There is also a distinction by altchain (for bitcoin testnet there is tpub / tprv). But when one intends to derive a public from a private there is a presumption that the derived public is on the same network. This requires a coupling of the network versions for the public-private key pairs. This requires external mapping information, which is generally hard-wired into the application.

To avoid hard-coding this mapping libbitcoin will require that HD private keys are initialized with both prefixes. This can't be derived deterministically because there are multiple altchain implementations in use and they are designed for cosmetic prefixes, and the domain is in flux. The internal prefix tuple will be parameterized as a 64 bit number with high and low 32 bit parts. In other words the size/length limitation will be preserved.

Defaults will be for mainnet, so for example testnet, like any other altchain, will require parameterization. This allows public and private keys to be derived along the proper chain once the private key is intialized. Public keys are initialized using only the 32 bit public prefix, since they cannot derive private keys.

The bx command line parameterization will allow this combined value, which will also be configurable in default wallet parameters in the config file.

[skaht]

I believe extended BIP 32 public or private keys should be agnostic of the network they are being applied to. In other words, BIP 32 public or private key prefixes should be independent of the cryptocurrency network they are being applied to. It is the 8-bit "--version" value that is critical for converting extended private/public key information for use on a specific cryptocurrency network, especially for altcoin specific WIF key, and address information. This will simplify the design of multi-currency HD wallets by removing unwanted complexity.

[evoskuil]

I don't necessarily disagree, but then of course something may need to be done about this... because what you are saying implies hard-coding the two prefixes.

[skaht]

I ran into that very same URL a few months back while investigating the application of bx to altcoins. I had developed the table below that indicated to me that customized BIP32 prefixes for each altcoin's public and private extended key is not a well-defined practice. If the specific altcoins developers want to break inter-operability with a generalized approach that can subsume numerous altcoins that is their prerogative. The KISS approach works for me. It will also be liked by wallet developers desiring to easily support numerous cryptographic currencies in a common application.

http://bitcoin.stackexchange.com/questions/38878/how-does-the-bip32-version-bytes-convert-to-base58/38890#38890                                                     
————————————————————————————————————————————————————————————————————————————————————————————————————
      |    BIP 44     |      mainnet      |      mainnet      |      testnet      |     testnet         
Coin  | EXT_COIN_TYPE |   EXT_SECRET_KEY  |   EXT_PUBLIC_KEY  |   EXT_SECRET_KEY  |  EXT_PUBLIC_KEY   
————————————————————————————————————————————————————————————————————————————————————————————————————
BTC   |  0x80000000   |  0x0488ADE4/xprv  |  0x0488B21E/xpub  |  0x04358394/tprv  |  0x043587CF/tpub  https://github.com/bitcoin/bitcoin/blob/master/src/chainparams.cpp
TEST  |  0x80000001   |  0x04358394/tprv  |  0x043587CF/tpub  |        N/A        |        N/A        https://github.com/bitcoin/bitcoin/blob/master/src/chainparams.cpp
                                                                                                      https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki
                                                                                                      https://github.com/bluedroplet/coins/blob/master/coins.yaml
LTC   |  0x80000002   |  0x0488ADE4/xprv  |  0x0488B21E/xpub  |  0x04358394/tprv  |  0x043587CF/tpub  https://github.com/litecoin-project/litecoin/blob/master-0.10/src/chainparams.cpp
LTC   |  0x80000002   |  0x019d9cfe/Ltpv  |  0x019da462/Ltub  |  0x0436ef7d/ttpv  |  0x0436f6e1/ttub  https://github.com/Bit-Wasp/bitcoin-lib-php/blob/master/src/BIP32.php
DOGE  |  0x80000003   |  0x0488E1F4/xqRp  |  0x0488C42E/xq71  |  0x043575A4/tpi7  |  0x0435D1DF/tqiQ  https://github.com/dogecoin/dogecoin/blob/master/src/chainparams.cpp
DOGE  |  0x80000003   |  0x02fac398/dgpv  |  0x02facafd/dgub  |  0x0432a243/tgpv  |  0x0432a9a8/tgub  https://github.com/Bit-Wasp/bitcoin-lib-php/blob/master/src/BIP32.php
RDD   |  0x80000004   |                   |                   |                   |                   
DASH  |  0x80000005   |  0x02FE52CC/drkp  |  0x02FE52F8/drkv  |  0x3A805837/DRKP  |  0x3A8061A0/DRKV  https://github.com/dashpay/dash/blob/master/src/chainparams.cpp
PPC   |  0x80000006   |                   |                   |                   |                     
NMC   |  0x80000007   |  0x0488ADE4/xprv  |  0x0488B21E/xpub  |  0x04358394/tprv  |  0x043587CF/tpub  https://github.com/domob1812/namecore/blob/master/src/chainparams.cpp
FTC   |  0x80000008   |                   |                   |                   |                
XCP   |  0x80000009   |                   |                   |                   |                 
BLK   |  0x8000000A   |  0x0488ADE4/xprv  |  0x0488B21E/xpub  |  0x04358394/tprv  |  0x043587CF/tpub  https://github.com/rat4/blackcoin/blob/master/src/chainparams.cpp
NSR   |  0x8000000B   |                   |                   |                   |                 
NBT   |  0x8000000C   |                   |                   |                   |                 
MZC   |  0x8000000D   |                   |                   |                   |                 
VIA   |  0x8000000E   |  0x0488ADE4/xprv  |  0x0488B21E/xpub  |  0x04358394/tprv  |  0x043587CF/tpub  https://github.com/viacoin/viacoin/blob/master/src/chainparams.cpp
XCH   |  0x8000000F   |                   |                   |                   |                 
RBY   |  0x80000010   |                   |                   |                   |                 
GRS   |  0x80000011   |                   |                   |                   |                 
DGC   |  0x80000012   |  0x0488ADE4/xprv  |  0x0488B21E/xpub  |  0x04358394/tprv  |  0x043587CF/tpub  https://github.com/digibyte/digibyte/blob/master/src/chainparams.cpp                                   
CCN   |  0x80000013   |                   |                   |                   |                 
DGB   |  0x80000014   |  0x0488ADE4/xprv  |  0x0488B21E/xpub  |  0x04358394/tprv  |  0x043587CF/tpub  https://github.com/digibyte/digibyte/blob/master/src/chainparams.cpp
OA?   |  0x80000015   |                   |                   |                   |                 

[evoskuil]

I think that the approach I've outlined won't cause any problems regardless. It will always default to using the bitcoin prefix tuple, but if someone wants to provide an altcoin prefix they will need to provide both parts when constructing the private key. There is no envisioned relationship to payment/stealth address versions, nor is there any relationship to encrypted keys (which do embed payment address info) or WIF keys (which are also payment address agnostic). The myriad of prefixes is actually quite annoying :).

[skaht]

With the current version of bx, I could synthesize any altcoin the BIP44 Altcoin Version Mapping Table using existing testnet tprv/tpub extended keys, even for Dash that has its own mainnet drkp/drkv convention.

If I'm on the same wavelength, instead of fixing the prefixes for extended private or public keys as I've suggested. You desire to leave the choice open for the wallet developer to pick the prefix. This will work, but if consumers want vendor product portability of raw extended keys they will have yet one more configuration setting to make, which will likely increase the number of product support helpdesk tickets.

Guess there is no one right answer. Not sure how to weight developer needs and wallet customer needs.

[evoskuil]

Yes, there is no limitation produced by allowing the user to specify a coin-specific prefix. But it does then allow the software to process prefixes according to user expectations.

BTW, the same exact logic holds for WIF prefixes. WIF carries a compression flag and a coin-specific prefix. But the underlying secret is usable in any chain. The same is true for payment addresses (allowing for the oddity of a coin-specific P2SH prefix). The underlying public key hash is usable in any chain. And of course this also holds for stealth address prefixes. The prefix is just an intended use mnemonic.

In fact the only base58-encoded artifact that embeds a chain relationship in the data (as opposed to the wrapper/metadata) is BIP-38, since it hashes the encoded payment address into the payloads. This is IMO a mistake, probably resulting from the faulty theory behind confirmation codes, but alas.

[skaht]

Agree

[evoskuil]

Internally libbitcoin::wallet works with both compressed and uncompressed public keys. This is also an intended use convention, since the values are logically the same - it's just a question of whether the intent is to hash the longer or shorter variant. WIF (private) keys embed the intended compression for the corresponding public key for the sole purpose of generating a payment address. So we have chain intent and compression intent all over the wallet code. I've been working to abstract away the conceptual ugliness so the developer doesn't get lost in this nonsense.

[skaht]

Ah, how history repeats itself...

Kind of reminds decades ago wrestling with simulation kinematics or sensor engines written by others that use both the English units and Metric units and getting caught up in all the freaken unit conversion factors. Do once in metric internally, and change the units to whatever the customer wants externally. A few years ago NASA trashed a Martian vehicle due to such an issue.

Not fun, but such cleaning up makes cleaner and more reliable code. It will be many many months before I can get to that level of detail.

[evoskuil]

Implemented.