bitcoinjs / bitcoinjs-lib

A javascript Bitcoin library for node.js and browsers.
MIT License
5.61k stars 2.08k forks source link

Newbie here, need some help, stuck on psbt transaction creation and signing #1543

Closed RuiSiang closed 4 years ago

RuiSiang commented 4 years ago

Didn't have problem with the output, but when processing input, kept showing "Error: Data for input key witness Utxo is incorrect: Expected {script: Buffer; Value: number}" and got .....". The problem is, I already got the appropriate format there.

Also, when signing, though with the proper private key, it kept popping "No transactions were signed" errors.

the input is a 2 of 3 multisig input

Please help, thanks

Here's my code for transaction creation

var p = 'https://api.smartbit.com.au/v1/blockchain/address/'+addr;
const resp = await fetch(p);
const r = await resp.json();
var tx = new bitcoin.Psbt({
  network: bitcoin.networks.bitcoin
});
var amount = r.address.confirmed.balance_int;
for(var j = 0; j < r.address.transactions.length; j++) {
  var scriptpubkey='';
  for (var k = 0; k < r.address.transactions[j].output_count; k++) {
    if (r.address.transactions[j].outputs[k].addresses[0]==addr) {
      scriptpubkey=r.address.transactions[j].outputs[k].script_pub_key.hex;
    }
  }
  await tx.addInput({
    hash: r.address.transactions[j].hash,
    index: j,
    witnessUtxo: {
      script: Buffer.from(scriptpubkey, 'hex'),
      value: r.address.transactions[j].output_amount_int,
    },
  });
  var tx_receiver=await txr.toHex();
}

Here's the code for signing

sign = function (message, key) {
  var psbt = bitcoin.Psbt.fromHex(message);
  var keyPair = bitcoin.ECPair.fromWIF(key);
  psbt.signAllInputs(keyPair);
  return psbt.toHex();
}
junderw commented 4 years ago
  1. Clean up your code before asking questions, please.
  2. You are very confused by the data given by the API. The index you pass to addInput should be the output index, not the index of the transaction in the list from the API. Also, transaction output amount is just an estimate from the API, you want to get the specific amount of the output you're using.

Using their unspent API will be easier for you to understand I think:

'https://api.smartbit.com.au/v1/blockchain/address/' + addr + '/unspent';
RuiSiang commented 4 years ago

Sorry about that, my apologies. I’ll do so next time. Is a redeem script or witness/nonwitness required for this sort of transactions? And how can I generate the redeem script /witness properly, because I kept getting type error though I am using buffer from hex, thanks.

junderw commented 4 years ago

Read through every example in this and let me know if you have questions.

https://github.com/bitcoinjs/bitcoinjs-lib/blob/master/test/integration/transactions.spec.ts

(It is TypeScript, but it reads similarly to JavaScript and you can usually convert TypeScript to JavaScript by just removing all the type annotations.)

RuiSiang commented 4 years ago

I read all examples already, the problem is whether I'm getting the inputs right from the api (such as the redeem script buffer part) and the error "Error: Data for input key witness Utxo is incorrect: Expected {script: Buffer; Value: number}" and got ....." that doesn't let me include redeem script in input. Sorry if you think my questions are dumb

junderw commented 4 years ago

witnessUtxo is not the input... you don't include the redeemScript there, you include it in the object passed to addInput on the same level as hash and index...

junderw commented 4 years ago

inputData is what you pass to addInput inputData.witnessUtxo.script is the scriptPubkey of the prevout inputData.witnessUtxo.value is the amount of satoshis from the prevout inputData.redeemScript is the redeemScript in the case where the scriptPubkey is a P2SH script

junderw commented 4 years ago

But if your P2SH doesn't use P2WSH or P2WPKH you shouldn't use witnessUtxo to begin with since it's not segwit.

RuiSiang commented 4 years ago

Thanks a ton. Finally understood everything now

RuiSiang commented 4 years ago

One last question, is the redeemscript same as scriptpubkey, or if not, how can I generate it? I used redeemScript : Buffer.from(script_pubkey, 'hex'), but the data for input key redeemscript is incorrect, expected buffer and got {"type":"Buffer","data":[blahblahblah]} error is still popping up. I'm pretty sure I got the buffer right though

SFzxc commented 4 years ago

@RuiSiang. is the redeemscript same as scriptpubkey.

ScriptPubkey(Locking Script) of P2SH is <Hash160> <ScriptHash> <OP_EQUAL> <redeem Script> is your custom script before Hash160. It mean hash160(redeemScript) == <ScriptHash>

ScriptSig = <signature> + <redeem Script>

RuiSiang commented 4 years ago

So how can I generate a 2 of 3 redeem script if I have all three pub keys as in this scenerio? The docs just generate one out of nowhere with payment.output.encode XD

junderw commented 4 years ago
// Context from not included above: keys is an Array of ECPairs
// m is a number of required signatures that is calculated by the type string passed.
// network is defaulted to bitcoin network if not passed, but this function uses regtest, so we pass regtest every time.

// splitType is an array of types ie. ['p2wpkh', 'p2sh'] which will loop the payment object through creating a p2wpkh and then envelope it inside a p2sh payment.

  let payment;
  splitType.forEach(type => {
    if (type.slice(0, 4) === 'p2ms') {
      payment = bitcoin.payments.p2ms({
        m,
        pubkeys: keys.map(key => key.publicKey).sort(),
        network,
      });
    } else if (['p2sh', 'p2wsh'].indexOf(type) > -1) {
      payment = bitcoin.payments[type]({
        redeem: payment,
        network,
      });
    } else {
      payment = bitcoin.payments[type]({
        pubkey: keys[0].publicKey,
        network,
      });
    }
  });

So try looking at that dynamic loop of type embedding, and create a payment for p2sh-p2ms.

The example I gave would be:

const payment = bitcoin.payments.p2sh({
  redeem: bitcoin.payments.p2wpkh({
    pubkey: Buffer.from('03230b8a3661b4a649d63c46cd3b5d9787f3a38562fe0d880ac02a4f94157e1276', 'hex'),
  },
}

console.log(`scriptPubkey is ${payment.output.toString('hex')}`)
console.log(`redeemScript is ${payment.redeem.output.toString('hex')}`)

Taking a look at how the payments API works might help.

junderw commented 4 years ago

p2sh redeem output is called redeemScript p2wsh redeem output is called witnessScript

junderw commented 4 years ago

isPoint checks if they are valid DER public keys. (33 bytes starting with 0x02 or 0x03 or 65 bytes starting with 0x04)

ECPair's publicKey attribute formats it correctly in the getter. But bitcoin related sites, apps, and libraries usually only use DER.

RuiSiang commented 4 years ago

I'm confident in that the pubkey is valid, it is 33 bytes and starts with 03 or 02, though the format is string, so I used buffer.from to transfer it to hex. But how can I convert a 33 byte buffer uint8array to isPoint format? aka Buffer [Uint8Array][3, 12, 11, 255, ...] to <Buffer 03 0c 0b ff ...> I have tried various methods, but it seems that it doesn't work somehow

junderw commented 4 years ago
> let uintarr = new Uint8Array([1, 2, 3])
> let buf = Buffer.from(uintarr)
> buf
<Buffer 01 02 03>
> uintarr
Uint8Array(3) [ 1, 2, 3 ]

Buffer.from seems to work fine.

junderw commented 4 years ago

Yes I understand the error.

The causes are one of:

  1. It's not a valid point.
  2. It's not a Buffer.

Perhaps your "Buffer" dependency is not a real Buffer but some sort of react-native-psuedo-patched-in-Buffer??? I know a lot of frameworks have drop in replacements for things.

RuiSiang commented 4 years ago

Thanks, got it now

KayBeSee commented 4 years ago

I am running into a similar issue with using Buffers in the BIP32Derivation path

const bufferMasterFingerprint = Buffer.from('4f60d1c9', 'hex');
    const bufferPubKey = Buffer.from('tpubDFyWQKWfMju7qNo77ZCkeY8dt1oJ9Bc7RTDm1X4hytQhhzDo3PHYw71medvSk6BdGBinA8i2wLUKCbLYf5HZ6ejaFSNN7CF8xee1nkDyWgW', 'hex');

psbt.addInput({
      hash: tx.ins[i].hash.reverse().toString('hex'), // you have to reverse the buffer bc of that one bug in bitcoin-core
      index: tx.ins[i].index,
      sequence: tx.ins[i].sequence,
      witnessUtxo: {
        script: Buffer.from(
          inputUtxo.scriptpubkey,
          'hex'
        ),
        value: inputUtxo.value
      },
      witnessScript: Buffer.from(WITNESS_SCRIPT, 'hex'),
      bip32Derivation: [{
        masterFingerprint: bufferMasterFingerprint,
        pubkey: bufferPubKey,
        path: "m/44'/1'/0'/0/0",
      }]
      // redeemScript?
    });

UnhandledPromiseRejectionWarning: Error: Data for input key bip32Derivation is incorrect: Expected { masterFingerprint: Buffer; pubkey: Buffer; path: string; } and got [{"masterFingerprint":{"type":"Buffer","data":[79,96,209,201]},"pubkey":{"type":"Buffer","data":[]},"path":"m/44'/1'/0'/0/0"}]

How did you resolve the issue @RuiSiang? I am using Node.js native, no react-native.

junderw commented 4 years ago

please read the tests.

https://github.com/bitcoinjs/bitcoinjs-lib/blob/c95e15de01519b30c5eeb0f89700d03e49afb0e1/test/integration/transactions.spec.ts#L563-L625

You are using the wrong data for your pubkey. and I'm not sure you are using the right data for your masterFingerprint.

IronHeartDan commented 1 year ago

Error: No inputs were signed at Psbt.signAllInputs

I have just started learning and finally after implementing the whole code this is the error I am struck from hours Please let me know what wrong am I doing.

import { networks, payments, Psbt } from 'bitcoinjs-lib';
import { ECPairFactory } from "ecpair";
import * as ecc from "tiny-secp256k1";
import axios from 'axios';

async function sendCrypto() {

    const walletA = {
        "address": "n4CjraHDsmEjRbmVhx4DPZBR8RYVzLsWEw",
        "privateKey": "fd4e1b29c5cdf713c8dd327f672ee6e42de2933c8e576c6d0254820e6efc999f",
        "publicKey": "02edcd780e1a9e2cc140de56590b9b091b3b6716818c72d7bd97e999cebb724657"
    }

    const walletB = {
        "address": "n2oBEU446uM3ReFepRbSBUSGioZtyBwhgv",
        "publicKey": "039766e72f469e591f3d6b1b67905fe94e66dafe8766064982edafdc75e0fcbeaa",
        "privateKey": "7aa6a3f444835006c033f1bff9080b5ef5b26ed7bee20df636f202d80459904f"
    }

    const txb = new Psbt({ network: networks.testnet })

    const ECPair = ECPairFactory(ecc)

    const keyPair = ECPair.fromPrivateKey(Buffer.from(walletA.privateKey, 'hex'));

    const payment = payments.p2sh({ m: 2, redeem: payments.p2wpkh({ pubkey: keyPair.publicKey, network: networks.testnet }) });

    const { address } = payment;
    const redeemScript = payment.redeem?.output;

    if (!redeemScript || !address) return null;

    const response = await axios.get(`https://api.blockcypher.com/v1/btc/test3/addrs/${walletA.address}/full`);

    const utxos = response.data.txs.flatMap((tx) => {
        let voutIndex = 0;
        return tx.outputs.map((output) => {
            const utxo = {
                txid: tx.hash,
                vout: voutIndex++,
                scriptPubKey: output.script,
                amount: output.value,
            };

            if (!output.spent_by) {
                return utxo;
            }
        });
    }).filter(Boolean);

    // console.log(utxos);
    // return

    for (const element of utxos) {
        txb.addInput({
            hash: element.txid,
            index: element.vout,
            witnessUtxo: {
                script: Buffer.from(element.scriptPubKey, 'hex'),
                value: element.amount
            },
            redeemScript: redeemScript
        });
    }

    const senderTotalBalance = response.data.balance

    const amountToSend = 0.001 * 100000000 // BTC to Satoshis
    const fee = 0.00001 * 100000000

    const change = senderTotalBalance - (amountToSend + fee)

    txb.addOutput({
        address: walletB.address,
        value: amountToSend
    });

    txb.addOutput({
        address: address,
        value: change
    });

    txb.signAllInputs(keyPair)
    txb.finalizeAllInputs()

    const rawHex = txb.extractTransaction().toHex();
    console.log(rawHex);

}

sendCrypto()
junderw commented 1 year ago
  1. You only have funds on walletA (which is a P2PKH script).
  2. You are trying to spend a P2SH-P2WPKH (by inserting witnessUtxo and redeemScript from the p2sh-p2wpkh address generation)
  3. You added m: 2 to P2SH payment fields when m is only available on p2ms (multisig) payments

Looking at your code, I can't tell what you're trying to do. It's all over the place.

IronHeartDan commented 1 year ago
  1. You only have funds on walletA (which is a P2PKH script).
  2. You are trying to spend a P2SH-P2WPKH (by inserting witnessUtxo and redeemScript from the p2sh-p2wpkh address generation)
  3. You added m: 2 to P2SH payment fields when m is only available on p2ms (multisig) payments

Looking at your code, I can't tell what you're trying to do. It's all over the place.

Yes, I am trying to send funds from A->B. I did understood that there is some difference between P2PKH and P2SH-P2WPKH but how do I do it properly ? And also removed m2

junderw commented 1 year ago

https://github.com/bitcoinjs/bitcoinjs-lib/blob/v6.1.0/test/integration/transactions.spec.ts#L86-L171

  1. For each addInput you need hash, index, nonWitnessUtxo
  2. Ignore the parts about toBase64 and fromBase64 and splitting up the signing.

Basically the only thing that is different from your code is what you are adding in the addInput part.

Also be careful, your change address is a P2SH-P2WPKH, and is not a P2PKH like walletA is.

KayBeSee commented 1 year ago

Additionally, I believe the way you assign vout on your inputs (incrementing on the loop) needs to be corrected.

The vout field on the input should be the UTXO's index on the previous transaction you are spending from. I think, in your case, you want to use tx_output_n from the API response.

I did a workshop on building a wallet with BitcoinJS that you might find helpful for explaining how to construct a transaction. Specifically you want to look at Part 3.

Repo: https://github.com/KayBeSee/tabconf-workshop Video: https://www.youtube.com/watch?v=Bwz2P2hPVpk

IronHeartDan commented 1 year ago

https://github.com/bitcoinjs/bitcoinjs-lib/blob/v6.1.0/test/integration/transactions.spec.ts#L86-L171

  1. For each addInput you need hash, index, nonWitnessUtxo
  2. Ignore the parts about toBase64 and fromBase64 and splitting up the signing.

Basically the only thing that is different from your code is what you are adding in the addInput part.

Also be careful, your change address is a P2SH-P2WPKH, and is not a P2PKH like walletA is.

Thanks for making me understand I have change the code accordingly

const payment = payments.p2pkh({ pubkey: keyPair.publicKey, network: networks.testnet });

    const { address } = payment;
    for (const element of utxos) {
        txb.addInput({
            hash: element.txid,
            index: element.vout,
            nonWitnessUtxo: Buffer.from(element.scriptPubKey, 'hex')
        });
    }
    txb.signAllInputs(keyPair)
    txb.finalizeAllInputs()

        const rawHex = txb.extractTransaction().toHex();
        console.log(rawHex);

throw new Error('Cannot read slice out of bounds'); This is what I am getting now

IronHeartDan commented 1 year ago

Additionally, I believe the way you assign vout on your inputs (incrementing on the loop) needs to be corrected.

The vout field on the input should be the UTXO's index on the previous transaction you are spending from. I think, in your case, you want to use tx_output_n from the API response.

I did a workshop on building a wallet with BitcoinJS that you might find helpful for explaining how to construct a transaction. Specifically you want to look at Part 3.

Repo: https://github.com/KayBeSee/tabconf-workshop Video: https://www.youtube.com/watch?v=Bwz2P2hPVpk

I was also thinking the same thing but the API response doesn't includes the index also looking at your video will let you know if it solves my problem

junderw commented 1 year ago

nonWitnessUtxo: Buffer.from(element.scriptPubKey, 'hex')

Please actually read the links I send you. Don't just look at the attribute name.

From the link I sent you:

    {
      const {
        hash, // string of txid or Buffer of tx hash. (txid and hash are reverse order)
        index, // the output index of the txo you are spending
        nonWitnessUtxo, // the full previous transaction as a Buffer
      } = inputData1;
      assert.deepStrictEqual({ hash, index, nonWitnessUtxo }, inputData1);
    }

"the full previous transaction as a Buffer" and nonWitnessUtxo: Buffer.from(element.scriptPubKey, 'hex') do not match.

IronHeartDan commented 1 year ago

Yes It works now Thank you so much for the help

This is the code for anyone in future

    for (const element of utxos) {
        const response = await axios.get(`https://api.blockcypher.com/v1/btc/test3/txs/${element.txid}?includeHex=true`);
        const previousTx = response.data.hex;
        txb.addInput({
            hash: element.txid,
            index: element.vout,
            nonWitnessUtxo: Buffer.from(previousTx, 'hex')
        });
    }
IronHeartDan commented 1 year ago

Error: Can not finalize input #0 So I was able to use this code and generated raw hash then I broadcasted it using https://live.blockcypher.com/btc/pushtx/ It was successful. I added more funds to WalletA using Faucent and again tried to run the code but ran into this error

import { networks, payments, Psbt } from 'bitcoinjs-lib';
import { ECPairFactory } from "ecpair";
import * as ecc from "tiny-secp256k1";
import axios from 'axios';

async function sendCrypto() {

    const walletA = {
        "address": "n4CjraHDsmEjRbmVhx4DPZBR8RYVzLsWEw",
        "privateKey": "fd4e1b29c5cdf713c8dd327f672ee6e42de2933c8e576c6d0254820e6efc999f",
        "publicKey": "02edcd780e1a9e2cc140de56590b9b091b3b6716818c72d7bd97e999cebb724657"
    }

    const walletB = {
        "address": "n2oBEU446uM3ReFepRbSBUSGioZtyBwhgv",
        "publicKey": "039766e72f469e591f3d6b1b67905fe94e66dafe8766064982edafdc75e0fcbeaa",
        "privateKey": "7aa6a3f444835006c033f1bff9080b5ef5b26ed7bee20df636f202d80459904f"
    }

    const txb = new Psbt({ network: networks.testnet })

    const ECPair = ECPairFactory(ecc)

    const keyPair = ECPair.fromPrivateKey(Buffer.from(walletA.privateKey, 'hex'));

    // const payment = payments.p2sh({ redeem: payments.p2wpkh({ pubkey: keyPair.publicKey, network: networks.testnet }) });
    // const redeemScript = payment.redeem?.output;
    // if (!redeemScript || !address) return null;

    const payment = payments.p2pkh({ pubkey: keyPair.publicKey, network: networks.testnet });

    const { address } = payment;

    const response = await axios.get(`https://api.blockcypher.com/v1/btc/test3/addrs/${walletA.address}/full`);

    const senderTotalBalance = response.data.balance

    const amountToSend = 0.001 * 100000000 // BTC to Satoshis
    const fee = 0.00001 * 100000000
    const deductions = (amountToSend + fee)
    const change = senderTotalBalance - deductions

    const utxos = response.data.txs.flatMap((tx) => {
        let voutIndex = 0;
        return tx.outputs.map((output) => {
            const utxo = {
                txid: tx.hash,
                vout: voutIndex++,
                scriptPubKey: output.script,
                amount: output.value,
            };

            if (!output.spent_by) {
                return utxo;
            }
        });
    }).filter(Boolean);

    const filteredUtxos = utxos.filter((utxo) => {
        return utxo.amount >= deductions;
    });

    for (const element of filteredUtxos) {
        const response = await axios.get(`https://api.blockcypher.com/v1/btc/test3/txs/${element.txid}?includeHex=true`);
        const previousTx = response.data.hex;
        txb.addInput({
            hash: element.txid,
            index: element.vout,
            nonWitnessUtxo: Buffer.from(previousTx, 'hex')
        });
    }

    txb.addOutput({
        address: walletB.address,
        value: amountToSend
    });

    txb.addOutput({
        address: address,
        value: change
    });

    try {

        txb.signAllInputs(keyPair)
        txb.finalizeAllInputs()

        const rawHex = txb.extractTransaction().toHex();
        console.log("Raw Hex");
        console.log(rawHex);

    } catch (error) {
        console.error(error);
    }

}

sendCrypto()