bitcore,Bitcoinjs vs pycoin

[shayanb]

I'm trying to implement a multisig structure between two currently running systems, one is using js (BitcoinJS for now but it could be switched to bitcore if needed) and the other using python, pycoin.

The issue is for pycoin to know the txIns of a transaction hex it needs to be exported using include_unspents flag, tx.as_hex(include_unspents=True) to have spendables attached to it. Now this hex is apparently meaningless to bitcoinjs.

Now as a fix I don't include unspents in the unsigned_tx generated by pycoin, send it to bitcoinjs, partially sign it there, send it back to pycoin and try to fully sign it and it fails.

So I tried the solution by lekanovic to set_unspents() on regular transactions (unsigned generated by bitcoinjs, signed by pycoin) and it works, but not on multisig.

when I tried to sign a partially signed transaction (with bitcoinjs) and fixed unspents, I get this error:

File "multisig/views.py", line 188, in sign_tx
    fully_signed = partially_signed_tx.sign(LazySecretExponentDB([cloud_priv_key], {}, netcode=COIN_NETWORK), p2sh_lookup=p2sh_lookup)
  File "pycoin/tx/Tx.py", line 397, in sign
    hash160_lookup, idx, self.unspents[idx].script, hash_type=hash_type, **kwargs)
  File "pycoin/tx/Tx.py", line 273, in sign_tx_in
    self.txs_in[tx_in_idx].script = self.solve(hash160_lookup, tx_in_idx, tx_out_script, hash_type=SIGHASH_ALL, **kwargs)
  File "pycoin/tx/Tx.py", line 269, in solve
    existing_script=self.txs_in[tx_in_idx].script, **kwargs)
  File "pycoin/tx/pay_to/ScriptPayToScript.py", line 41, in solve
    underlying_solution = script_obj.solve(**kwargs)
  File "pycoin/tx/pay_to/ScriptMultisig.py", line 101, in solve
    sig_pair, actual_signature_type = parse_signature_blob(data)
  File "pycoin/tx/script/check_signature.py", line 41, in parse_signature_blob
    sig_pair = der.sigdecode_der(sig_blob[:-1], use_broken_open_ssl_mechanism=True)
  File "pycoin/tx/script/der.py", line 111, in sigdecode_der
    rs_strings, empty = remove_sequence(sig_der)
  File "pycoin/tx/script/der.py", line 62, in remove_sequence
    len(string), binascii.hexlify(string[:10])))
UnexpectedDER: wanted sequence (0x30), got string length 64 04eb1e0ccd9afcac4222

[shayanb]

This line is causing the issue: https://github.com/richardkiss/pycoin/blob/master/pycoin/tx/script/der.py#L59 @richardkiss any ideas why?

[richardkiss]

See if this patch helps you:

--- a/pycoin/tx/pay_to/ScriptMultisig.py
+++ b/pycoin/tx/pay_to/ScriptMultisig.py
@@ -1,5 +1,6 @@
 from ..script import opcodes, tools
 from ..script.check_signature import parse_signature_blob
+from ..script.der import UnexpectedDER
 from ..script.microcode import VCH_TRUE

 from ... import ecdsa
@@ -109,7 +110,7 @@ class ScriptMultisig(ScriptType):
                             existing_signatures.append(data)
                             secs_solved.add(sec_key)
                             break
-                    except encoding.EncodingError:
+                    except (encoding.EncodingError, UnexpectedDER):
                         # if public_pair is invalid, we just ignore it
                         pass

[richardkiss]

This is in master now as def9d3a640319aa284b015dd892e20848098716a

[shayanb]

I think that solved that specific issue, but still it's impossible to fully sign a partially signed transaction from BitcoinJS nor bitcore with pycoin:

Using the tx hex as is results in:

  File "multisig.py", line 479, in <module>
    bitcoinjs_partiallysigned_test()
  File "multisig.py", line 436, in bitcoinjs_partiallysigned_test
    fully_signed0 = Tx.from_hex(partially_signed_bitcoinjs).sign(LazySecretExponentDB([prv_2nd_key], {}, netcode=COIN_NETWORK), p2sh_lookup=p2sh_lookup, netcode=COIN_NETWORK)
  File "multisig/virtualenv/lib/python2.7/site-packages/pycoin/tx/Tx.py", line 433, in sign
    self.check_unspents()
  File "multisig/virtualenv/lib/python2.7/site-packages/pycoin/tx/Tx.py", line 388, in check_unspents
    raise ValueError("wrong number of unspents. Call unspents_from_db or set_unspents.")
ValueError: wrong number of unspents. Call unspents_from_db or set_unspents.

and fixing the txins by iterating over the spendables would result in:

  File "multisig.py", line 438, in bitcoinjs_partiallysigned_test
    fully_signed = fixed_tx.sign(LazySecretExponentDB([prv_2nd_key], {}, netcode=COIN_NETWORK), p2sh_lookup=p2sh_lookup, netcode=COIN_NETWORK)
  File "multisig/virtualenv/lib/python2.7/site-packages/pycoin/tx/Tx.py", line 440, in sign
    hash160_lookup, idx, self.unspents[idx].script, hash_type=hash_type, **kwargs)
  File "multisig/virtualenv/lib/python2.7/site-packages/pycoin/tx/Tx.py", line 277, in sign_tx_in
    self.txs_in[tx_in_idx].script = self.solve(hash160_lookup, tx_in_idx, tx_out_script, hash_type=SIGHASH_ALL, **kwargs)
  File "multisig/virtualenv/lib/python2.7/site-packages/pycoin/tx/Tx.py", line 256, in solve
    b2h(hash160))
ValueError: hash160=5629021f7668d4ec310ac5e99701a6d6cf95eb8f not found in p2sh_lookup

[shayanb]

it might has to do with dummy_signature that pycoin adds and bitcoinJS and bitcore does not.

[richardkiss]

Some sample code would be very helpful. Can you include the code that produces the output above?

[shayanb]

This is what produced the output above:

def bitcoinjs_partiallysigned_test(COIN_NETWORK="XTN"):
    partially_signed_bitcoinjs = "01000000014fca9a365b5318a6b1d2909375371ce8413ad414477fef19d22755b9a7edffda01000000fd16010047304402205319016d0fc0e334d1f09c8d52f037002a20a48f9a25202cd207aecb6d9a112c02204bb37f768a05c4e52e2627bebd7e84b1645e6765621cb4bfd5609579f2ae4f690100004cc95241048aa0d470b7a9328889c84ef0291ed30346986e22558e80c3ae06199391eae21308a00cdcfb34febc0ea9c80dfd16b01f26c7ec67593cb8ab474aca8fa1d7029d4104cf54956634c4d0bdaf00e6b1871c089b7a892d0fecc077f03b91e8d4d146861b0a4fdd237891a9819c878984d4b123f6fe92d9bbc05873a1bb4fe510145bf369410471843c33b2971e4944c73d4500abd6f61f7edf9ec919c408cbe12a6c9132d2cb8ebed8253322760d5ec6081165e0ab68900683de503f1544f03816d47fec699a53aeffffffff02a08601000000000017a9145629021f7668d4ec310ac5e99701a6d6cf95eb8f87a2df43000000000017a9145629021f7668d4ec310ac5e99701a6d6cf95eb8f8700000000"
    prv_2nd_key = "93VQSefMyfcxyx5FsV9czruYF4ddom9wAVzXnbyzCCRvdYznCSj"
    raw_script = ["5241048aa0d470b7a9328889c84ef0291ed30346986e22558e80c3ae06199391eae21308a00cdcfb34febc0ea9c80dfd16b01f26c7ec67593cb8ab474aca8fa1d7029d4104cf54956634c4d0bdaf00e6b1871c089b7a892d0fecc077f03b91e8d4d146861b0a4fdd237891a9819c878984d4b123f6fe92d9bbc05873a1bb4fe510145bf369410471843c33b2971e4944c73d4500abd6f61f7edf9ec919c408cbe12a6c9132d2cb8ebed8253322760d5ec6081165e0ab68900683de503f1544f03816d47fec699a53ae"]
    p2sh_lookup=build_p2sh_lookup(raw_script)

    second_key_spendables = spendables_for_addr("2N16oE62ZjAPup985dFBQYAuy5zpDraH7Hk", netcode="XTN")
    try:
        fully_signed0 = Tx.from_hex(partially_signed_bitcoinjs).sign(LazySecretExponentDB([prv_2nd_key], {}, netcode=COIN_NETWORK), p2sh_lookup=p2sh_lookup, netcode=COIN_NETWORK)
        print "non fixed fully sign: %s" %fully_signed0.as_hex()
    except Exception, E:
        print "non fixed failed : %s" %E
        fixed_tx = fix_tx(partially_signed_bitcoinjs, second_key_spendables)
        print "fixed tx: %s" %fixed_tx
        fully_signed = fixed_tx.sign(LazySecretExponentDB([prv_2nd_key], {}, netcode=COIN_NETWORK), p2sh_lookup=p2sh_lookup, netcode=COIN_NETWORK)
        print "fullysigned: %s" %fully_signed

and what I do to fix the unspents a tx hex.

def fix_tx(hex_tx, spendables):
    print "fixing tx"
    tx = Tx.from_hex(hex_tx)
    unspents = []
    for txin in tx.txs_in:
        for s in spendables:
            if txin.previous_hash == s.tx_hash and txin.previous_index == s.tx_out_index:
                print "txin fixed: %s" % b2h(s.tx_hash)
                sp = Spendable(s.coin_value,
                            s.script,
                            txin.previous_hash,
                            txin.previous_index)
                unspents.append(sp)

    tx.set_unspents(unspents)
    return tx

[shayanb]

So basically the solution for now would be to sign with pycoin first, then send the tx.as_hex(include_unspents = False) to bitcoinJS to fully sign the transaction.

[Zibbo]

I think the problem that causes: ValueError: hash160=5629021f7668d4ec310ac5e99701a6d6cf95eb8f not found in p2sh_lookup

is that build_p2sh_lookup expects scripts to be in a binary format. Yours is hex encoded. Try:

raw_script = [h2b("5241048aa0d470b7a9328889c84ef0291ed30346986e22558e80c3ae06199391eae21308a00cdcfb34febc0ea9c80dfd16b01f26c7ec67593cb8ab474aca8fa1d7029d4104cf54956634c4d0bdaf00e6b1871c089b7a892d0fecc077f03b91e8d4d146861b0a4fdd237891a9819c878984d4b123f6fe92d9bbc05873a1bb4fe510145bf369410471843c33b2971e4944c73d4500abd6f61f7edf9ec919c408cbe12a6c9132d2cb8ebed8253322760d5ec6081165e0ab68900683de503f1544f03816d47fec699a53ae")]
p2sh_lookup=build_p2sh_lookup(raw_script)
h2b("5629021f7668d4ec310ac5e99701a6d6cf95eb8f") in p2sh_lookup

True

[richardkiss]

This should be fixed in 34d4230682417723268e3dfda3fbfce31803e10a