I am not being able to sign PSBT transactions with Zprv

[FeatureSpitter]

I am just toying around with this in my Arduino ESP32 Nano, and for some reason I am not being able to export a valid PSBT when using a private key in the Zprv format, the following way:

#include "Bitcoin.h"
#include "PSBT.h"

#include "Secrets.h"

#define MAX_PSBT_SIZE 3000

HDPrivateKey hd(XPRV); // Just a string like "ZprvAhrnh(...)"
PSBT psbt;

void setup() {
  Serial.begin(115200);
  while (!Serial) ;
  Serial.println(LABEL);
  Serial.println("Please input the base 64 PSBT:");
}

void loop() {
  if (Serial.available() > 0) {

    String input = "";
    char c;
    int charCount = 0;

    while (true) {
      c = Serial.read();
      if(c == '\n' || charCount >= MAX_PSBT_SIZE) {
        break;
      } else if ((int)c < (int)'!' || (int)c > (int)'z') {
        continue;
      }

      input += c;
      charCount++;
    }

    Serial.println("Received PSBT:");
    Serial.println(input);
    // Parse the input as base64 PSBT
    psbt.parseBase64(input);

    // Check parsing is OK
    if (!psbt) {
      Serial.println("Failed parsing transaction");
      Serial.println("Please input the base 64 PSBT:");
      return;
    }

    Serial.println("Transactions details:");

    // going through all outputs to print info
    Serial.println("Outputs:");
    for(int i=0; i<psbt.tx.outputsNumber; i++){
      // print addresses
      Serial.print(psbt.tx.txOuts[i].address(&Regtest));
      Serial.print(" -> ");
      Serial.print(psbt.tx.txOuts[i].btcAmount()*1e3);
      Serial.println(" mBTC");
    }
    Serial.print("Fee: ");
    Serial.print(float(psbt.fee()));
    Serial.println(" sats");

    psbt.sign(hd);
    Serial.println("============================================");
    Serial.println(psbt.toBase64()); // now you can combine and finalize PSBTs in Bitcoin Core
    Serial.println("============================================");

    Serial.println(LABEL);
    Serial.println("Please input the base 64 PSBT:");
    return;
  }
}

This basically receives a PSBT from the serial comm, and outputs the signed PSBT.

However, I believe the parsing of Zprv is buggy, I tried to debug the fingerprint part and the parsed fingerprint seems totally different, then from that point on everything fails, the PSBT.sign function cannot find a fit public key matching my private key and proceeds without signing.

PS. Forget "Secrets.h", it just defines the XPRV constant (I should actually call it ZPRV).

[FeatureSpitter]

For example, for this key:

(Notice the Master fingerprint of 5A:21:88:34).

Then I export this with Sparrow to the Electrum json format to obtain the xprv:

{
  "keystore": {
    "xpub": "zpub6rgGh4Q21dLzrac8fqhtCubP5JpfDSf6BJLyJKjfn3D2ojp56PFn7nD7pwVRMtJk9EkkFDW4UrGeDWyFnBfptgvC8uvjWY6PoutNMLwzTyu",
    "xprv": "zprvAY3WiCK4YLmi4neGTqnauNMd8cFLhobtcP3ANMi4CtbLeCeVRzRWCw9qivm3stUnXQ3hwDfCSBzY5qUWcLeatCjadMLmaebwR2fS2suNaHL",
    "type": "bip32",
    "pw_hash_version": 1
  },
  "wallet_type": "standard",
  "use_encryption": false,
  "seed_type": "bip39"
}

Now, if I provide

HDPrivateKey hd("zprvAY3WiCK4YLmi4neGTqnauNMd8cFLhobtcP3ANMi4CtbLeCeVRzRWCw9qivm3stUnXQ3hwDfCSBzY5qUWcLeatCjadML");

The fingerprint I will find at this part of the code will be: B9:40:6D:D9

And in this part of the code will be: FF:73:D5:35

And as you can see from the images above, they have nothing to do with the real master fingerprint of 5A:21:88:34.

I have no idea how the initally parsed B9:40:6D:D9 becomes FF:73:D5:35 when it reaches the PSBT signing function, but both values do not match the expected 5A:21:88:34.

Thus, I believe there's a bug parsing these Zprv keys. Maybe even parsing other parts as well, I didn't validate further because the fingerprint alone is reason for it to fail.

It makes the PSBT.sign() function fail at this condition, thus ignore signing any PSBT, because it will never match any fingerprint in the PSBT metadata.

[FeatureSpitter]

@stepansnigirev Can you have a look plz?

[FeatureSpitter]

I made this drafty parser in node just for sanity check, that this key is valid:

const bs58check = require('bs58check')

function arrayToHex(array) {
  return Array.from(array, byte => uint8ToHex(byte));
}

function uint8ToHex(byte) {
  return ('0' + (byte & 0xFF).toString(16).toUpperCase()).slice(-2);
}

function uint32ToHex(byte) {
  return new Array(uint8ToHex(byte >> 24) ,
          uint8ToHex(byte >> 16) ,
          uint8ToHex(byte >> 8) ,
          uint8ToHex(byte >> 0));
}

const decodeExtendedKey = (key) => {
  const decoded = bs58check.decode(key)

  return {
    receivedSequence: arrayToHex(decoded),
    version: decoded.slice(0, 4),
    depth: uint8ToHex(Buffer.from(decoded.slice(4, 5)).readUInt8(0)),
    parentFingerprint: arrayToHex(decoded.slice(5, 9)),
    childNumber: uint32ToHex(Buffer.from(decoded.slice(9, 13)).readUInt32BE(0)),
    chainCode: arrayToHex(decoded.slice(13, 45)),
    key: arrayToHex(decoded.slice(45))
  }
}

const parseVersion = (version) => {
  switch(arrayToHex(version).join('')) {
    case '02AA7A99':
      return 'Zprv';
    case '02AA7ED3':
      return 'Zpub';
    case '04B2430C':
      return 'zprv';
    case '04B24746':
      return 'zpub';
    default:
      return 'Unknown';
  }
}

const key = 'zprvAY3WiCK4YLmi4neGTqnauNMd8cFLhobtcP3ANMi4CtbLeCeVRzRWCw9qivm3stUnXQ3hwDfCSBzY5qUWcLeatCjadMLmaebwR2fS2suNaHL'
const decodedKey = decodeExtendedKey(key)
const version = parseVersion(decodedKey.version)

console.log(`Version: ${version}`)
console.log(`Depth: ${decodedKey.depth}`)
console.log(`Parent Fingerprint: ${decodedKey.parentFingerprint}`)
console.log(`Child Number: ${decodedKey.childNumber}`)
console.log(`Chain Code: ${decodedKey.chainCode}`)
console.log(`Key: ${decodedKey.key}`)
console.log(`Received Sequence: ${decodedKey.receivedSequence}`)

The output is as expected, however the fingerprint is still not the one expected:

Version: zprv
Depth: 00
Parent Fingerprint: B9,40,6D,D9
Child Number: 80,00,00,00
Chain Code: 83,DB,ED,BE,2D,BF,D2,68,FA,50,88,9D,DD,45,F8,EA,98,A4,05,D8,5C,7E,94,6B,6D,93,F2,8E,AD,54,39,5C
Key: 00,78,1A,74,D0,AA,4F,64,94,14,9A,52,E8,70,40,1C,51,29,C8,30,33,51,3C,78,30,88,D6,E2,81,EB,14,0A,AB
Received Sequence: 04,B2,43,0C,00,B9,40,6D,D9,80,00,00,00,83,DB,ED,BE,2D,BF,D2,68,FA,50,88,9D,DD,45,F8,EA,98,A4,05,D8,5C,7E,94,6B,6D,93,F2,8E,AD,54,39,5C,00,78,1A,74,D0,AA,4F,64,94,14,9A,52,E8,70,40,1C,51,29,C8,30,33,51,3C,78,30,88,D6,E2,81,EB,14,0A,AB

[stepansnigirev]

I think the reason is that PSBT provides derivation path for the root master private key, and you are trying to sign with zprv that is already derived from the root with derivation m/84'/0'/0'. Therefore the library can not detect that the key you are trying to sign can actually sign - derivation paths and fingerprints don't match.

In other words, PSBT has derivations like m/84'/0'/0'/0/0 that is a valid derivation for the root key, but zprv would need only the last part m/0/0.

Automatic signing of the PSBT works only with the root key, so either sign with the root key, or change derivation paths provided in the PSBT to include only the part from zprv (drop first 3 indexes).

You can generate root key from the mnemonic and password like this:

HDPrivateKey hd("habit opera fox human grow relax snow shoulder just knife tail guilt", "mypassword!");

You can also make your own custom signing function. For reference see sign method in PSBT

[FeatureSpitter]

Hello @stepansnigirev that was my very first approach, assuming that by password you mean the passphrase, also known as the 25th word among the 24 mnemonic words. It wasn't able to sign as well.

What is that password argument exactly?

[stepansnigirev]

The password argument is passphrase in the screenshot above.

Also the zprv in the backup is quite strange - it has zero depth but non-zero parent fingerprint.

Would be nice if you can provide full test vector:

1. mnemonic and passphrase
2. corresponding zprv from the backup
3. test psbt transaction that you are trying to sign

Then I can reproduce it and see what is wrong.

If you are working on mainnet you can DM me the keys on Telegram.

[FeatureSpitter]

@stepansnigirev maybe you missed this reply above?

https://github.com/micro-bitcoin/uBitcoin/issues/31#issuecomment-1773704736

[FeatureSpitter]

I don't have the psbt, because the one I tested was using my own wallet to which I can't share the private key.

Also, it is a multisig, in which one of the keys is in the hardware wallet.

If you can make it work with the mnemonic of my reply above (notice its passphrase), you can try creating a wsh(sortedmulti(2,Keystore1,Keystore2)) (i.e. a P2WSH 2 of 2 multisig) to mimic my scenario. Then you create a tx, sign it with Keystore1, then produce the PSBT as base64 in Sparrow and try to sign it with uBitcoin (you can use my code above that already receives the PSBT as b64 via Serial comm).

But before setting up this more complex scenario, try first with the mnemonic above.

[stepansnigirev]

Did you try signing with the key derived like this?

HDPrivateKey hd("video large practice token obvious border charge prize sustain night smoke payment rubber child pulp agree ankle turtle abstract human style ensure limit race", "xxxxxx");

You can set up sparrow on testnet and reproduce it there - then I hope it won't be an issue to share PSBT.

So far I checked that zprv in the backup is indeed derived from the root key with derivation m/84'/0'/0', and then for some reason the depth is changed to zero.

[FeatureSpitter]

So, I've created a 2/2 multisig with these keys in sparrow:

With these keys:

This is the psbt I've created to sign:

cHNidP8BAKQCAAAAAyTA5seUQXomJWqjx72R/cVwioChm0EDIiKCe/ufuYZ8AQAAAAD9////abHWFrF9o245EPIAVU38XOjXne2CxU5lNGI+ssauC2kBAAAAAP3///879kHNFQxJ90zQfyjpaRr59MdmvKCeCGedYVTezkBIGgAAAAAA/f///wEOPQAAAAAAABYAFHKxH5uJMDKjFhUbRX1RYU1b5MISPa0mAE8BBDWHzwQGqJI0gAAAAhN4L30bW26HQXQFyObZr9NciRWeq3ozB5w1879hr8njAoW/7qT5ajpy9603cyhTYIkTEOnmP6FI6Qm7mAOuww6XFCZk+FkwAACAAQAAgAAAAIACAACATwEENYfPBMUmv5qAAAACfKKqF6cIBuH1wN4rFqmcvy5KFn6kivlYbZicKfiDHjMCjwGWbAi0v6pcxZZFaduoh7wO+2RRUgcvOy0eC4GUYjoUyan0kzAAAIABAACAAAAAgAIAAIAAAQB9AgAAAAEERhNz0bsQkdY6DlVTkSvoPHZ9YveyrNi8ZhgsE6iEgQIAAAAA/f///wKObRcAAAAAABYAFGPVGIGdWRuFeKrX3IMsltvyV5NFQB8AAAAAAAAiACBJuMarHSPQyRuuAAtXWOB0/nhZHxLdhuTJM00A0wnpkzmtJgABAStAHwAAAAAAACIAIEm4xqsdI9DJG64AC1dY4HT+eFkfEt2G5MkzTQDTCemTAQMEAQAAAAEFR1IhAhzeigRmZ7/jrca9EUCwQThzRAihKRFpL2XO0PtOPUbfIQJI+ZYPX7sTaIsUrO6P+4N4FsUJIXCGnfk3WD2XlSP+UFKuIgYCSPmWD1+7E2iLFKzuj/uDeBbFCSFwhp35N1g9l5Uj/lAcJmT4WTAAAIABAACAAAAAgAIAAIAAAAAAAAAAACIGAhzeigRmZ7/jrca9EUCwQThzRAihKRFpL2XO0PtOPUbfHMmp9JMwAACAAQAAgAAAAIACAACAAAAAAAAAAAAAAQB9AgAAAAGj6RQwgTIGoY3iv44PJUHXhyY4VHChnI+t7uhjXOO2YQEAAAAA/f///wKlJQUAAAAAABYAFCSutEuy7twOTzdz9dL2A7YVRm2d6AMAAAAAAAAiACBJuMarHSPQyRuuAAtXWOB0/nhZHxLdhuTJM00A0wnpkzqtJgABASvoAwAAAAAAACIAIEm4xqsdI9DJG64AC1dY4HT+eFkfEt2G5MkzTQDTCemTAQMEAQAAAAEFR1IhAhzeigRmZ7/jrca9EUCwQThzRAihKRFpL2XO0PtOPUbfIQJI+ZYPX7sTaIsUrO6P+4N4FsUJIXCGnfk3WD2XlSP+UFKuIgYCSPmWD1+7E2iLFKzuj/uDeBbFCSFwhp35N1g9l5Uj/lAcJmT4WTAAAIABAACAAAAAgAIAAIAAAAAAAAAAACIGAhzeigRmZ7/jrca9EUCwQThzRAihKRFpL2XO0PtOPUbfHMmp9JMwAACAAQAAgAAAAIACAACAAAAAAAAAAAAAAQB9AgAAAAE/MDy7c9OUuFEvTN1exgn1ZG7kKB8Z1vKNvRdeRw9GpwAAAAAA/f///wIyGwAAAAAAACIAIEm4xqsdI9DJG64AC1dY4HT+eFkfEt2G5MkzTQDTCemT060IAAAAAAAWABRDKsiuan8tzgiu5MezsTBiATCjjTqtJgABASsyGwAAAAAAACIAIEm4xqsdI9DJG64AC1dY4HT+eFkfEt2G5MkzTQDTCemTAQMEAQAAAAEFR1IhAhzeigRmZ7/jrca9EUCwQThzRAihKRFpL2XO0PtOPUbfIQJI+ZYPX7sTaIsUrO6P+4N4FsUJIXCGnfk3WD2XlSP+UFKuIgYCSPmWD1+7E2iLFKzuj/uDeBbFCSFwhp35N1g9l5Uj/lAcJmT4WTAAAIABAACAAAAAgAIAAIAAAAAAAAAAACIGAhzeigRmZ7/jrca9EUCwQThzRAihKRFpL2XO0PtOPUbfHMmp9JMwAACAAQAAgAAAAIACAACAAAAAAAAAAAAAAA==

This is uBitcoin signed psbt output:

cHNidP8BAKQCAAAAAyTA5seUQXomJWqjx72R/cVwioChm0EDIiKCe/ufuYZ8AQAAAAD9////abHWFrF9o245EPIAVU38XOjXne2CxU5lNGI+ssauC2kBAAAAAP3///879kHNFQxJ90zQfyjpaRr59MdmvKCeCGedYVTezkBIGgAAAAAA/f///wEOPQAAAAAAABYAFHKxH5uJMDKjFhUbRX1RYU1b5MISPa0mAAAiAgIc3ooEZme/463GvRFAsEE4c0QIoSkRaS9lztD7Tj1G30cwRAIgYHEbUWXM7GZMZpTA2DiKugIZlf8xvpAaIfIbsbG3fC4CIDEwLJ9z5V1skKG43/ogvu/7CQFGgQ/ULeNepVu23WY9AQAiAgIc3ooEZme/463GvRFAsEE4c0QIoSkRaS9lztD7Tj1G30gwRQIhAOXcgt9lmW9MulHuVWXBbLLYx9dsowKHux69fALX4ajgAiAW4KStDW5ds6R6hrMfuqMMAbySAKu8pguPfmH2Mu6yaAEAIgICHN6KBGZnv+Otxr0RQLBBOHNECKEpEWkvZc7Q+049Rt9HMEQCIEpI4UMDG8hdvpQi7qQTZO2ca9qAPvZNwEbDQUIGxERzAiBR/DJZrulLpzTtiAZsgFUN366eSfb5yYLrkxIb2rcKFAEAAA==

And this is my code:

#include "Bitcoin.h"
#include "PSBT.h"

#define MAX_PSBT_SIZE 3000

HDPrivateKey hd("six similar sketch beauty play depend absorb lawn private harsh target gown pyramid crawl devote hill total source crew quantum lawn veteran brass way","xxxxxx");
PSBT psbt;

void setup() {
  Serial.begin(115200);
  while (!Serial) ;
  Serial.println("Please input the base 64 PSBT:");
}

void loop() {
  if (Serial.available() > 0) {
    String input = "";
    char c;
    int charCount = 0;

    while (true) {
      c = Serial.read();
      if(c == '\n' || charCount >= MAX_PSBT_SIZE) {
        break;
      } else if ((int)c < (int)'!' || (int)c > (int)'z') {
        continue;
      }

      input += c;
      charCount++;
    }

    Serial.println("Received PSBT:");
    Serial.println(input);
    // Parse the input as base64 PSBT
    psbt.parseBase64(input);

    // Check parsing is OK
    if (!psbt) {
      Serial.println("Failed parsing transaction");
      Serial.println("Please input the base 64 PSBT:");
      return;
    }

    Serial.println("Transactions details:");

    // going through all outputs to print info
    Serial.println("Outputs:");
    for(int i=0; i<psbt.tx.outputsNumber; i++){
      // print addresses
      Serial.print(psbt.tx.txOuts[i].address(&Regtest));
      Serial.print(" -> ");
      Serial.print(psbt.tx.txOuts[i].btcAmount()*1e3);
      Serial.println(" mBTC");
    }
    Serial.print("Fee: ");
    Serial.print(float(psbt.fee()));
    Serial.println(" sats");

    psbt.sign(hd);
    Serial.println("============================================");
    Serial.println(psbt.toBase64()); // now you can combine and finalize PSBTs in Bitcoin Core
    Serial.println("============================================");

    Serial.println("Please input the base 64 PSBT:");
    return;
  }
}

It seems to be able to sign with this one. I'm going to try again with my mainnet 2/2 wallet for sanity, this is odd.

[FeatureSpitter]

It is working now for my mainnet one as well. I have tried to trace back my steps to see what I am doing differently. But this seems to be my mistake and not a bug.

Sorry for opening this false positive, I'll close it now.

Thanks!

Awesome project btw :)

[FeatureSpitter]

Btw @stepansnigirev I think I found a mistake on this example:

https://github.com/micro-bitcoin/uBitcoin/blob/master/examples/psbt/psbt.ino#L16C13-L16C13

I believe if (!psbt.parseBase64(input)) { is the one returning the result you expect, and not if(!psbt) {, because psbt is a memory reference, but parseBase64() returns the parsed length.

Or am I missing something?

[stepansnigirev]

psbt also contains a bool opeartor that returns true if psbt was parsed correctly, and fasle in case of parsing error.

[FeatureSpitter]

That's odd, because sometimes it was parsing my input correctly and sometimes it wasnt.

But then when I started using the parseBase64 result in the if it worked all the time.

No idea if it was me messing up, or if there is something odd with the parsing fuction.