Creating and signing the P2WSH-over-P2SH transaction

impowski commented 6 years ago

I'm facing a problem with creating a transaction P2WSH-over-P2SH, it's not a bug or anything, I just can't figure it out seeking for some help.

So here is the problem, I have a mutlisig wallet with 2 private keys on testnet.

p_key1 = 'b4741e2e87c19256c81a0662480b75f15831c7017d28b4586882c52b89f91bcf'
p_key2 = '3c212d5724a2a6bd5004c502a4be7ba55a1cf4a0923ffd1a9bb3cb6fd7201bed'

Then I use this function to generate a P2shScript:

def p2sh_script(pub_key_1, pub_key_2):
    witness_script = MultisigScript(2, pub_key_1, pub_key_2, 2)
    witness_script_hash = witness_script.p2wsh_hash()
    redeem_script = P2wshV0Script(witness_script_hash)
    script_pub_key = P2shScript(redeem_script)
    return (witness_script, redeem_script, script_pub_key)

It works fine, gives me the right address with which I take out unspents via API call.

And here is the 3 functions which I'm kinda confused about and can't figure what exactly I did wrong, I know for a fact that after signing a transaction I don't have anything in scriptSig in one of the inputs, I tried to play it around and read a documentation on P2WSH-over-P2SH in different sources, but that didn't helped me implement it with your library. Maybe you could help me out on that one @SimoneBronzini

def build_transaction(utxos, to, amount):
    witness_script = address['witness_script']
    redeem_script = address['redeem_script']
    script_pubkey = address['script_pubkey']

    # goes into input as a witness
    print(f'Witness Script for address `{script_pubkey.address()}`:\n{witness_script.hexlify()}')

    # goes into input as a script
    print(f'Redeem Script for address `{script_pubkey.address()}`:\n{redeem_script.hexlify()}')

    # goes to output which will be the final value of address
    print(f'Script Pub Key for address `{script_pubkey.address()}`:\n{script_pubkey.hexlify()}')

    ins, total = add_inputs(utxos, amount)

    # Generating output script for transaction
    to_send_address = P2shAddress.from_string(to)
    to_send_address_hash = to_send_address.hash
    to_send_script_pubkey = P2shScript(to_send_address_hash)

    # goes to output for amount which will get
    print(f'Script Pub Key for address `{to_send_address}`:\n{to_send_script_pubkey.hexlify()}')
    address['to_script'] = to_send_script_pubkey

    outs = []
    outs.append(
        TxOut(
            value=amount,
            n=0,
            script_pubkey=to_send_script_pubkey
        )
    )

    fee = address['fee']

    if total > amount + fee:
        outs.append(
            TxOut(
                value=total-amount-fee,
                n=1,
                script_pubkey=script_pubkey
            )
        )

    tx = MutableTransaction(
        version=0,
        ins=ins,
        outs=outs,
        locktime=Locktime(0)
    )

    return tx

def send():
    amount = currency_to_satoshi_cached(100, 'usd')
    fee = get_fee_cached()
    address['fee'] = fee
    # Get unspents
    utxos = get_utxos()
    #print(f'UTXOS: {utxos}')
    # Create a new unsigned transaction
    unsigned_tx = build_transaction(utxos, patau_wallet, amount)
    print(f'Unsigned Transaction: { unsigned_tx.hexlify() }')

    # Create a MultisigSolver for our address
    multisig_solver = MultisigSolver(address['pkey1'], address['pkey2'])

    # Create a P2shSolver for our P2shScript, which will solve our MultisigScript
    witness_solver = P2wshV0Solver(address['witness_script'], multisig_solver)
    script_solver = P2shSolver(address['redeem_script'], witness_solver)

    # TODO Sign an unsigned transaction
    signed_tx = unsigned_tx.spend(unsigned_tx.outs, [witness_solver, script_solver])
    print(f'Signed Transaction: { signed_tx.hexlify() }')

    # TODO Broadcast signed transaction to network
    # NetworkAPI.broadcast_tx_testnet(signed_tx.hexlify())

def add_inputs(unspents, amount):
    ins = []
    total = 0
    for unspent in unspents:
        script = ScriptSig.empty()
        txid = unspent.txid
        tx_index = unspent.txindex
        uns_amount = unspent.amount

        total += uns_amount
        ins.append(TxIn(
            txid=txid,
            txout=tx_index,
            script_sig=script,
            sequence=Sequence.max()
        ))

    return (ins, total)

impowski commented 6 years ago

I've managed somehow to get script_pubkey in both inputs but then witness is not how it's supposed to be in one of the inputs.

SimoneBronzini commented 6 years ago

I did not get into the details of your code, but in general creating and signing a p2wsh-over-p2sh transaction should be done along these lines:

# CREATION
p_key1 = PrivateKey.unhexlify('b4741e2e87c19256c81a0662480b75f15831c7017d28b4586882c52b89f91bcf')
p_key2 = PrivateKey.unhexlify('3c212d5724a2a6bd5004c502a4be7ba55a1cf4a0923ffd1a9bb3cb6fd7201bed')
witness_script = MultisigScript(2, p_key1.pub(), p_key2.pub(), 2)
redeem_script = P2wshV0Script(witness_script)
script_pubkey = P2shScript(redeem_script)

# SIGNING
solver = P2shSolver(redeem_script, P2wshV0Solver(witness_script, MultisigSolver(p_key1, p_key2)))
signed_tx = unsigned_tx.spend([tx.outs[0]], [solver])
# or, if you have many inputs and solvers you should go like
signed_tx = unsigned_tx.spend([tx1.outs[i], tx2.outs[k], tx3.outs[j]], [solver1, solver2, solver3])

I guess the wrong part of your code is:

signed_tx = unsigned_tx.spend(unsigned_tx.outs, [witness_solver, script_solver])

the spend method takes as input a list of outputs and a list of solvers, ordered such that output 0 is solved by solver 0, output 1 is solved by solver 1 and so on and so forth. In case you have any other doubts/problems please let me know.

impowski commented 6 years ago

@SimoneBronzini I didn't figure what exactly tx.outs[0] means? Is it different from unsigned_tx.outs which I generated?

SimoneBronzini commented 6 years ago

Yeah, the spend function takes as inputs the list of txouts you are spending (i.e. one output for each of your inputs), not the list of outputs of your new transaction, plus a solver for each output you are spending. If you want further details about this, you can find them in this README section.

impowski commented 6 years ago

@SimoneBronzini Ok, so if I have 2 unspents and there is basically a 2 inputs and 2 outputs, so I'll need to create 2 new Transactions or TxOuts which I then put into spend function? And if those 2 inputs from the same address so my solver will be the same for both transactions?

SimoneBronzini commented 6 years ago

Let's say you have two TxOuts, txout0 and txout1, that you want to spend in a new transaction:

solver0 = ...  # solves txout0.script_pubkey
solver1 = ...  # solves txout1.script_pubkey
unsigned_tx = ... # transaction with two inputs, one per txout you want to spend
signed = unsigned_tx.spend([txout0, txout1], [solver0, solver1])

of course, if the scriptPubKeys of the two txouts are the same, you can as well use the same solver, but still you'll have to pass it twice:

solver = ...  # solves both txout0.script_pubkey and txout1.script_pubkey
unsigned_tx = ...  # transaction with two inputs, one per txout you want to spend
signed = unsigned_tx.spend([txout0, txout1], [solver, solver])

impowski commented 6 years ago

@SimoneBronzini So here is the thing, I did everything like you described, created 2 transactions and took needed txouts and I'm getting the wrong witness on one of the inputs.

Here is the hex of transaction:

010000000001021b119ace084fd2bdb0bb687389f6f51346481ba5646a1eeec3fe70750a09452c0100000023220020d99be0d7e677f86ac50fe8a1e833cb1ef3e99539f9337f08469deed67b649c6affffffffb0cbb5ac31474f3de5128ca60e59c483510bee13d720fa01a928588aef04e74b0000000023220020d99be0d7e677f86ac50fe8a1e833cb1ef3e99539f9337f08469deed67b649c6affffffff02208c17000000000017a9141f6b27ac1ad977dc69642248e5e765fc526d24d887ed102f350000000017a91469356515093228913b1a518214b492288647ab7887040048304502210087b99ede4c708f7aeca9e217acbce486c533ae6b26e95154c2f213b9b2b62ac80220078d0792391dae13ebc7abe8c5eb71fbd20bdfee281c5c90635c508d5f8911b701483045022100b968d47e48b3b938b3e5b17ca336080be452302cca339947ebf5baa8fd854f9c02202a146abcaeaaa5c595540f91c777c07cf73dae28d4cc06c649b3109f2948d85201475221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae0400483045022100e82aa517eac442bdc656403e5d8ed684ccee485ee6779d10fa1e816bf7faeab5022018bb41553763765811b27226e91236bc7fe61415a6a87827db13bc67330d5b4a01483045022100d3e0e9e0b623e7b2f15b29b3ff25efa419d421af0429cda9fc46779e65e0aa4c022043095259f44e8e55291f8e80a4e7f8c17cd695f34f1cc6bf83b9d25cd7d1d0b301475221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae00000000

SimoneBronzini commented 6 years ago

Can you post the following data:

txouts you are trying to spend (with their scriptPubKey, redeemScript, witnessScript where applicable)
unsigned tx that spends those outs
signed tx obtained through the spend method (which I guess is the one you posted above)
data needed by both txouts to be spent (sighashes, private keys plus any other needed data, of course post them only if you don't care about losing the money you keep on those private keys)
expected scriptSig
expected witness

impowski commented 6 years ago

@SimoneBronzini TxOuts:

[TxOut(value=50000000, n=0, scriptPubKey='OP_HASH160 69356515093228913b1a518214b492288647ab78 OP_EQUAL')

TxOut(value=843820211, n=1, scriptPubKey='OP_HASH160 69356515093228913b1a518214b492288647ab78 OP_EQUAL')]

UsignedTx:

01000000021b119ace084fd2bdb0bb687389f6f51346481ba5646a1eeec3fe70750a09452c0100000000ffffffffb0cbb5ac31474f3de5128ca60e59c483510bee13d720fa01a928588aef04e74b0000000000ffffffff0268b217000000000017a9141f6b27ac1ad977dc69642248e5e765fc526d24d887bfea2e350000000017a91469356515093228913b1a518214b492288647ab788700000000

SignedTx:

010000000001021b119ace084fd2bdb0bb687389f6f51346481ba5646a1eeec3fe70750a09452c0100000023220020d99be0d7e677f86ac50fe8a1e833cb1ef3e99539f9337f08469deed67b649c6affffffffb0cbb5ac31474f3de5128ca60e59c483510bee13d720fa01a928588aef04e74b0000000023220020d99be0d7e677f86ac50fe8a1e833cb1ef3e99539f9337f08469deed67b649c6affffffff0268b217000000000017a9141f6b27ac1ad977dc69642248e5e765fc526d24d887bfea2e350000000017a91469356515093228913b1a518214b492288647ab78870400483045022100ec0aa92b888bf43e6caf6cafde53219a0bc132c3f8db565c6d73371a9945b4c902202a37b2838942d64356489b434a3e64897712174a3d3a0df98e7e4d9e19e72dd5014730440220428f0bab416b1ce091c8fce45c6e80f8dd283af58d8cd4ba2ef4daa1e65ce72b0220402762b7d4f6a9d137eb905e8131b5bf5e1a8dc261efeb7222f518cf0b710b7f01475221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae040047304402203a503824afc700a95068c0bdc8906c85ab68d37ff932ee9133685c93e4298d3a0220641b4b8050710606842c49ed812f13690309e0a5e0cab3916be7de355eb34afc014730440220087e81af4c2bb71810f7051a2af93edb8c561043ac4adb2c915bfe27e5fa315f022060ad3b2c5478bdf9f2374e29681dc06dca4f7acc9848599b2c13a48fd0d5542701475221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae00000000

PrivateKeys in Hex:

p_key1 = 'b4741e2e87c19256c81a0662480b75f15831c7017d28b4586882c52b89f91bcf'
p_key2 = '3c212d5724a2a6bd5004c502a4be7ba55a1cf4a0923ffd1a9bb3cb6fd7201bed'

WitnessScript:

5221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae

RedeemScript:

0020d99be0d7e677f86ac50fe8a1e833cb1ef3e99539f9337f08469deed67b649c6a

ScriptPubKey:

a91469356515093228913b1a518214b492288647ab7887

Expected Witness on inputs:

"304402203a503824afc700a95068c0bdc8906c85ab68d37ff932ee9133685c93e4298d3a0220641b4b8050710606842c49ed812f13690309e0a5e0cab3916be7de355eb34afc01", 
"30440220087e81af4c2bb71810f7051a2af93edb8c561043ac4adb2c915bfe27e5fa315f022060ad3b2c5478bdf9f2374e29681dc06dca4f7acc9848599b2c13a48fd0d5542701",
"5221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae"

Expected ScriptSig:

220020d99be0d7e677f86ac50fe8a1e833cb1ef3e99539f9337f08469deed67b649c6a

SimoneBronzini commented 6 years ago

Thank you, will look into this as soon as possible!

SimoneBronzini commented 6 years ago

Hi, sorry for the late answer. I tried the following:

>>> ws=ScriptBuilder.identify('5221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae')
>>> rs=P2wshV0Script(ws)       
>>> spk=P2shScript(rs)         
>>> ws.hexlify() == '5221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae'
True
>>> rs.hexlify() == '0020d99be0d7e677f86ac50fe8a1e833cb1ef3e99539f9337f08469deed67b649c6a'
True
>>> spk.hexlify() == 'a91469356515093228913b1a518214b492288647ab7887'
True
>>> uns = Transaction.unhexlify('01000000021b119ace084fd2bdb0bb687389f6f51346481ba5646a1eeec3fe70750a09452c0100000000ffffffffb0cbb5ac31474f3de5128ca60e59c483510bee13d720fa01a928588aef04e74b0000000000ffffffff0268b217000000000017a9141f6b27ac1ad977dc69642248e5e765fc526d24d887bfea2e350000000017a91469356515093228913b1a518214b492288647ab788700000000')
>>> p_key1 = PrivateKey.unhexlify('b4741e2e87c19256c81a0662480b75f15831c7017d28b4586882c52b89f91bcf')
>>> p_key2 = PrivateKey.unhexlify('3c212d5724a2a6bd5004c502a4be7ba55a1cf4a0923ffd1a9bb3cb6fd7201bed')
>>> solv = P2shSolver(rs, P2wshV0Solver(ws, MultisigSolver(p_key1, p_key2)))
>>> signed = uns.to_mutable().spend([TxOut(value=50000000, n=0, script_pubkey=Script.compile('OP_HASH160 69356515093228913b1a518214b492288647ab78 OP_EQUAL')), TxOut(value=843820211, n=1, script_pubkey=Script.compile('OP_HASH160 69356515093228913b1a518214b492288647ab78 OP_EQUAL'))], [solv, solv])
>>> str(signed)                
'SegWitTransaction(version=1, ins=[TxIn(txid=2c45090a7570fec3ee1e6a64a51b484613f5f6897368bbb0bdd24f08ce9a111b, txout=1, script_sig=0020d99be0d7e677f86ac50fe8a1e833cb1ef3e99539f9337f08469deed67b649c6a, sequence=4294967295, witness=Witness(["", "3045022100ec0aa92b888bf43e6caf6cafde53219a0bc132c3f8db565c6d73371a9945b4c902202a37b2838942d64356489b434a3e64897712174a3d3a0df98e7e4d9e19e72dd501", "30440220428f0bab416b1ce091c8fce45c6e80f8dd283af58d8cd4ba2ef4daa1e65ce72b0220402762b7d4f6a9d137eb905e8131b5bf5e1a8dc261efeb7222f518cf0b710b7f01", "5221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae"])), TxIn(txid=4be704ef8a5828a901fa20d713ee0b5183c4590ea68c12e53d4f4731acb5cbb0, txout=0, script_sig=0020d99be0d7e677f86ac50fe8a1e833cb1ef3e99539f9337f08469deed67b649c6a, sequence=4294967295, witness=Witness(["", "304402203a503824afc700a95068c0bdc8906c85ab68d37ff932ee9133685c93e4298d3a0220641b4b8050710606842c49ed812f13690309e0a5e0cab3916be7de355eb34afc01", "30440220087e81af4c2bb71810f7051a2af93edb8c561043ac4adb2c915bfe27e5fa315f022060ad3b2c5478bdf9f2374e29681dc06dca4f7acc9848599b2c13a48fd0d5542701", "5221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae"]))], outs=[TxOut(value=1553000, n=0, scriptPubKey=\'OP_HASH160 1f6b27ac1ad977dc69642248e5e765fc526d24d8 OP_EQUAL\'), TxOut(value=892267199, n=1, scriptPubKey=\'OP_HASH160 69356515093228913b1a518214b492288647ab78 OP_EQUAL\')], locktime=Locktime(0))'
>>> '220020d99be0d7e677f86ac50fe8a1e833cb1ef3e99539f9337f08469deed67b649c6a' == '220020d99be0d7e677f86ac50fe8a1e833cb1ef3e99539f9337f08469deed67b649c6a'
True
>>> ("304402203a503824afc700a95068c0bdc8906c85ab68d37ff932ee9133685c93e4298d3a0220641b4b8050710606842c49ed812f13690309e0a5e0cab3916be7de355eb34afc01",
... "30440220087e81af4c2bb71810f7051a2af93edb8c561043ac4adb2c915bfe27e5fa315f022060ad3b2c5478bdf9f2374e29681dc06dca4f7acc9848599b2c13a48fd0d5542701",
... "5221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae") == \
... ("304402203a503824afc700a95068c0bdc8906c85ab68d37ff932ee9133685c93e4298d3a0220641b4b8050710606842c49ed812f13690309e0a5e0cab3916be7de355eb34afc01",
... "30440220087e81af4c2bb71810f7051a2af93edb8c561043ac4adb2c915bfe27e5fa315f022060ad3b2c5478bdf9f2374e29681dc06dca4f7acc9848599b2c13a48fd0d5542701",
... "5221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae")
True

apparently the only thing that differs between what you expect and what btcpy produces is that with btcpy the witness begins with an additional element: the "" that you can see at the beginning of the witness:

Witness(["", "304402203a503824afc700a95068c0bdc8906c85ab68d37ff932ee9133685c93e4298d3a0220641b4b8050710606842c49ed812f13690309e0a5e0cab3916be7de355eb34afc01", "30440220087e81af4c2bb71810f7051a2af93edb8c561043ac4adb2c915bfe27e5fa315f022060ad3b2c5478bdf9f2374e29681dc06dca4f7acc9848599b2c13a48fd0d5542701", "5221020d2036fea368d0400acabe96e975b6f4e24c6f40f2e45db4264c2f117702c1e72102d0b84f4a3d4890b2fdde4dc74db4328fb3504e0247187d8286311c75b3b6bdeb52ae"])

That element is actually needed as it is the OP_0 required at the beginning of multisig transactions spending data.

chainside / btcpy

Creating and signing the P2WSH-over-P2SH transaction #41