Bip32_extract_key() output add always "01" at the end in the result

[gabridome]

When I extract the private key from a BIP32 wallet the result is longer than the correct result by two hex character always "01". when I try to derive the public key the resulting key is the same I would obtain from the correct private key.

Using key utility by Richard Kiss:

>ku xprv9s21ZrQH143K4XcG9k2UAQVxyDDFPcaqoum6fvjjfw8WypRy9nz1wBhFLNKponDYWsNwtgbGLqkt5uXBcW9QuAatAV5yuuGQ8aV72yiFSQw
sec as hex: 48dbef33ef7ed682ed3adf37a8ade74dd9636e18732ca8cbb486d016f86678e6

using bip32_extract_key:

>>>bip32_extract_key('xprv9s21ZrQH143K4XcG9k2UAQVxyDDFPcaqoum6fvjjfw8WypRy9nz1wBhFLNKponDYWsNwtgbGLqkt5uXBcW9QuAatAV5yuuGQ8aV72yiFSQw')
48dbef33ef7ed682ed3adf37a8ade74dd9636e18732ca8cbb486d016f86678e601

the weird thing is that if I use privtopub('48dbef33ef7ed682ed3adf37a8ade74dd9636e18732ca8cbb486d016f86678e601') the result is the same I find with the correct resulting private key (48dbef33ef7ed682ed3adf37a8ade74dd9636e18732ca8cbb486d016f86678e6) i.e: 02f2f4f20aaad94e3d2f309630f81ba2aec193adbc4b30e3d5d1c0f1c403b9ae3b

but if I try to derive the public from the "01" added one with bitaddress.org the result is: 03EA2558411BC9055B9EFF95F1B250FFA4798B6F876EF4BBD15320CAF206C5425B

Version: bitcoin-1.1.35

[wizardofozzie]

The 01 is the flag for a compressed private key, since it's impossible to know whether this hex representation is for compressed (02/03) or uncompressed (04). I'm not sure how bitaddress works, but pybitcointools drops the 01 with privtopub

[gabridome]

Ah ok. Now I see. Is it a standard representation or it is just a pybitcointools one?

It is good to know when you use the resulting keys with other libraries or you have to import them into a wallet.

[wizardofozzie]

Is it a standard representation or it is just a pybitcointools one?

@gabridome It's a standard representation, not just a pybitcointools one. I'll have to take a look at how bip32 handles addresses, but from what I can tell, the addresses are always compressed.

The reason bitaddress.org is won't accept 48dbef33ef7ed682ed3adf37a8ade74dd9636e18732ca8cbb486d016f86678e6**01** is because the accepted formats (below) doesn't have HEXC (hex compressed)

Key Formats: WIF, WIFC, HEX, B64, B6, MINI, BIP38

[fcracker79]

That format comes from base58 checked serialized format.

If you base58check deserialize the extended private key, you will notice that you do not get any '01'. It is added in bip32_deserialize. Simply put, it produces the base58check deserialized key from a WIF format, and specifically, always compressed. It is legal on WIF format, but not exactly correct for a private key. See also https://en.bitcoin.it/wiki/Wallet_import_format, point 2.

The compressed/uncompressed attribute should come up when extracting public keys and addresses from a private key. Probably pybitcointools should require a 'compressed' boolean parameter and rip that '01'.

[wizardofozzie]

@fcracker79

and specifically, always compressed

Are all bip32 private keys compressed WIFs, or is private ckd able to return an uncompressed WIF?

it produces the base58check deserialized key from a WIF format,

To clarify, are you referring to the WIF xprv format?

If the tool @ bip39.org is correct, then it's the former (compressed WIFs).

Boolean compressed parameter

Ideally, yeah, in fact I only became aware that the appended 01 could be used to signify which pubkey the SEC hex privkey was referring to. IMO, it's quite useful, since functions like decode_privkey look for length

[fcracker79]

Are all bip32 private keys compressed WIFs, or is private ckd able to return an uncompressed WIF?

All the bip32 private keys generated by the library are "compressed". That is hardcoded.

To clarify, are you referring to the WIF xprv format?

No, I am referring to the standard WIF format. The xprv format is a normal base58check serialization. What pybitcointools does is extract the key from the 'xprv' base58 string and add a '01' byte. As an example:

m = bip32_master_key(b'ciao')
-> 'xprv9s21ZrQH143K4aaJUtsu6UMgzXdrWc4RxR7qd9bkaVZTsUTr5kZFtKM5zxD48WFLnf1sZp1amZpZ4Cejd5kGn4ygEsoKr9NTT93tP4p3fV6'

# Notice the last 01 byte
bip32_extract_key(m)
-> ceef5dc641fe698f18bc481c3e7736143b531873311e632015c8206722f9abf801

# Notice that you do not have that key in the deserialized xprv form.
# But if you rip off the last 01 byte, you can find your private key at position 93
hexlify(changebase(m, 58, 256))
-> 0488ade4000000000000000000fd3686804569bf09e1e8325a7fa4270adb15e25b6c7b4ffa38b0523f25b64dd400*ceef5dc641fe698f18bc481c3e7736143b531873311e632015c8206722f9abf8*bff6efd5

# Compressed public key
privtopub('ceef5dc641fe698f18bc481c3e7736143b531873311e632015c8206722f9abf801')
-> 0215e98954bbf80792f885168f04d2a25b2a59addc3de1c69a43a1a1970cd18401

# Again, the very same compressed public key
bip32_extract_key(bip32_privtopub(m))
-> 0215e98954bbf80792f885168f04d2a25b2a59addc3de1c69a43a1a1970cd18401

decode_privkey look for length

All in all, pybitcointools is consistent with the convention of keeping the compression flag together with the private key. Provided that you are aware of that, you should not come into trouble.