P2WPKH signature size can vary, affecting transactions with large inputs

Problem

The signatures for P2WPKH inputs in a Bitcoin transaction are standard ECDSA signatures, but they are separated and placed in the SegWit witness data. Like ECDSA signatures, P2WPKH signatures utilize DER encoding, which results in their byte length varying between 71 and 72 bytes.

Read more: DER Encoding - Learn Me A Bitcoin

This minor size difference in signatures typically does not pose a problem when the Bitcoin transaction includes only a few inputs. In our implementation, it allows for an estimated fee variability of ±1 satoshi, meaning that a transaction that pays 1 satoshi more or less is acceptable:

https://github.com/ckb-cell/rgbpp-sdk/blob/ebcefc3322de4487a0951da49d82a123103b9073/packages/btc/src/transaction/build.ts#L218-L223

However, when constructing a large transaction with numerous inputs, this variability can cause issues. Each P2WPKH signature may differ by 1 byte, making it challenging to accurately predict the estimated fee. For example, when signing a transaction with 1,000 inputs, the total size difference in signatures could range from 0 to 1,000 bytes.

Resolver

To address the issue, we can refactor the fee estimation process.

Instead of signing the transaction with a random key pair and then calculating the fee based on the signed transaction, we can add a step at the end to check each input's type and status, identifying all the DER-encoded signatures:

const psbt = await this.createEstimatedPsbt();
const tx = psbt.extractTransaction(true);

let derEncodings = 0;
let witnessSizeShift = 0;
psbt.data.inputs.forEach((input, index) => {
  if (input.witnessUtxo) {
    const script = input.witnessUtxo.script;
    if (isP2wpkhScript(script)) {
      derEncodings += 1;
      const witness = tx.ins[index].witness;
      if (witness[0].byteLength > 71) {
        witnessSizeShift -= 1;
      }
    }
  }
});

Next, we can calculate the minimum and maximum expected fee range for the transaction:

const withWitnessSize = tx.byteLength(true);
const withoutWitnessSize = tx.byteLength(false);
const witnessSize = withWitnessSize - withoutWitnessSize;

const minWitnessSize = witnessSize + witnessSizeShift;
const maxWitnessSize = witnessSize + witnessSizeShift + derEncodings;
const minFee = this.calculateFeeWithSizes(withoutWitnessSize, minWitnessSize, feeRate);
const maxFee = this.calculateFeeWithSizes(withoutWitnessSize, maxWitnessSize, feeRate);
return {
  min: minFee,
  max: maxFee,
};

So when paying fee before the transaction is actually signed by the user, we can simply require the transaction to pay at least the maximum expected fee from the fee range we just calculated, so in this way, the transaction fee is always fulfilled:

const expectedFeeRange = await this.calculateFeeRange(currentFeeRate);
currentFee = expectedFeeRange.max;

const paidFee = this.summary().inputsRemaining;
isFeeExpected = paidFee >= currentFee;

Overpayment Issue

The resolver mentioned above leads to another problem: if the calculation or estimation of the transaction fee somehow goes wrong and results in a fee that is significantly larger than expected, we lack a clear logic to check its validity. For example, if the maximum expected fee is 10,000, what should we do if it paid 11,000? Or worse, what should we do if it paid 2,000?

We may need to validate the transaction's fee rate as well, ensuring that the difference between the expected fee rate and the actual paid fee rate does not exceed 1. i.e. if the expected fee rate is 3, the maximum paid fee rate should be around:

3 (expected fee rate) <= actual fee rate <= 4 (max acceptable fee rate)

Or, when updating the isFeeExpected, just make sure the paid fee equals to the current fee:

isFeeExpected = paidFee === currentFee;

utxostack / rgbpp-sdk