What is the public_hex, why is it needed when adding inputs?

[arshbot]

Hey there!

Confused as to why I need to include ki.public_hex when I'm adding an input? I've gone digging through the code and I see keys is optional, but I'm unable to sign and get an error posted below. I'd like to know why this is and if there's a way for me to construct the transaction fully first and then sign it.

>>> t = Transaction(network='dash_testnet')
>>> prev_tx = "5b5903a9e5f5a1fee68fbd597085969a36789dc5b5e397dad76a57c3fb7c232a"
>>> ki = Key('cTuDU2P6AhB72ZrhHRnFTcZRoHdnoWkp7sSMPCBnrMG23nRNnjUX', network='dash_testnet', compressed=False)
>>> t.add_input(prev_hash=prev_tx, output_n=0, keys=ki.public_hex, compressed=False)
0
>>> t.add_output(99900000, 'yUV8W2RmEbKZD8oD7YMeBNiydHWmormCDj')
0
>>> t.sign(ki.private_byte)

Error thrown if ki.public_hex is not provided

bitcoinlib.transactions.TransactionError: This key does not sign any known key: b'\x04\xc6\xee8\xb8\xd2\x89\xda;\x02\x08!\xc1\x89\x80&X\x85@\x89\xbd[\xed\xcd\xe9,\xae\x06f&\x80\xc1\x86\xd2\xd44\x9e0//- \xaa\xa7\x83p5"1\x8bB\xdc\xaaJ\xc9eS\xc1\xa0\xbb\'D\xe0\xd0_'

[mccwdev]

Hi, To create a transaction input at least a public key is needed. The public key is optional because the key can also be extracted from the unlocking script if provided, for example if you import raw transactions.

[arshbot]

It seems like it's the public key of the private key, which would translate to being the unformatted address? I guess I'm a bit confused since other libraries (like bitcoinjs, bitcore) don't require this information for input, only tx hash and index.

[mccwdev]

The public key of a transaction input can be found on the blockchain, so I assume the other libraries retrieve the public key from there. However in this library the Transaction object does not retrieve blockchain information, you will need to use the Wallet or Service class for that

[arshbot]

Ah I see, you're talking about the scriptpubkey?

[arshbot]

Can you provide an example transaction creation using the scriptpubkey?

[mccwdev]

Please ignore my previous statement. Public keys are not found on the blockchain, only public key hashes. At this moment it is not possible to create a signable transaction in BitcoinLib with a public key hash or scriptpubkey only a key (private or public).

As workaround you can provide the public key instead of the full private when creating the transaction:

 ki = Key('cTuDU2P6AhB72ZrhHRnFTcZRoHdnoWkp7sSMPCBnrMG23nRNnjUX',  network='dash_testnet', compressed=False)
 ki_public = ki.public()
 t = Transaction(network='dash_testnet')
 prev_tx = "5b5903a9e5f5a1fee68fbd597085969a36789dc5b5e397dad76a57c3fb7c232a"
 t.add_input(prev_hash=prev_tx, output_n=0, keys=ki_public, compressed=False)
 t.add_output(99900000, 'yUV8W2RmEbKZD8oD7YMeBNiydHWmormCDj')

But it should be possible with a public key hash or scriptpubkey / locking script. Will look into this

[arshbot]

Do you know what the ki.public() or ki.public_hex represents? When you say public key, are we talking about the public key used to derive the address?

The problem I'm having is that my system crafts transactions in one place and then signs the serialized transaction in another secure place. Right now this library requires me to perform construction in the secure place as adding inputs requires sensitive data. Do you know of a way to complete construction without exposing the price key?

[mccwdev]

Both ki.public() and ki.public represent the public key hex. From the public key you derive the public key hash: ki.hash160(), and the address is just a representation of the public key hash. Both public key and public key hash can be safely shared without security risk, so you can create transaction in a less secure place.

You could also use the wallet class for this. The wallet class manages keys and addresses and for instance creates a new address for every transaction. See https://github.com/1200wd/bitcoinlib/tree/master/examples for some examples

[mccwdev]

I've made some changes to the Input class to address the original issue when no keys are provided when creating a transaction.

See commit https://github.com/1200wd/bitcoinlib/commit/17df3aea614a1cc4f4f5ad4c8973d5003bbc70c8 and https://github.com/1200wd/bitcoinlib/commit/5fa38f0dc0391eed4f346f2ccc9b16f977f4a43e in the Segwit branch

[arshbot]

Thanks a ton! This really helps us out :)

I'll add a working example here so it's clear to others how to go about this. Thanks again!

[arshbot]

@mccwdev

Just installed the segwit-support branch to give it a go - ran into this error

>>> from bitcoinlib.transactions import *

>>> t = Transaction()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/harshagoli/.local/share/virtualenvs/python_playgrounds-lxx_pTfd/src/bitcoinlibsegwit/bitcoinlib/transactions.py", line 1193, in __init__
    self.network = Network(network)
  File "/Users/harshagoli/.local/share/virtualenvs/python_playgrounds-lxx_pTfd/src/bitcoinlibsegwit/bitcoinlib/networks.py", line 164, in __init__
    self.prefix_bech32 = NETWORK_DEFINITIONS[network_name]['prefix_bech32']
KeyError: 'prefix_bech32'

'prefix_bech32'

Following Create and sign transaction with add_input, add_output methods

[arshbot]

Fix looks trivial, just need to add this line to networks.py. Any reason this shouldn't work? If not, I'll take the opportunity to contribute 😄

[mccwdev]

It looks like there is an old config file in .bitcoinlib/config/networks.json. If you remove the .bitcoinlib/log/install.log file, all config files will be recreated, which should solve this problem.

[arshbot]

Couldn't find the file you're talking about (.bitcoinlib/log/install.log) in the project I forked, or the working version installed by pipenv. Here's a tree.

(python_playgrounds-lxx_pTfd) ➜  python_playgrounds tree /Users/harshagoli/.local/share/virtualenvs/python_playgrounds-lxx_pTfd/src/bitcoinlibfork/
/Users/harshagoli/.local/share/virtualenvs/python_playgrounds-lxx_pTfd/src/bitcoinlibfork/
├── LICENSE
├── MANIFEST.in
├── README.rst
├── __init__.py
├── bitcoinlib
│   ├── __init__.py
│   ├── config
│   │   ├── __init__.py
│   │   ├── opcodes.py
│   │   └── secp256k1.py
│   ├── data
│   │   ├── networks.json
│   │   └── providers.json
│   ├── db.py
│   ├── encoding.py
│   ├── keys.py
│   ├── main.py
│   ├── mnemonic.py
│   ├── networks.py
│   ├── services
│   │   ├── __init__.py
│   │   ├── authproxy.py
│   │   ├── baseclient.py
│   │   ├── bitcoind.py
│   │   ├── bitcoinlibtest.py
│   │   ├── bitgo.py
│   │   ├── blockchaininfo.py
│   │   ├── blockchair.py
│   │   ├── blockcypher.py
│   │   ├── blockexplorer.py
│   │   ├── blocktrail.py
│   │   ├── chainso.py
│   │   ├── coinfees.py
│   │   ├── cryptoid.py
│   │   ├── dashd.py
│   │   ├── estimatefee.py
│   │   ├── litecoind.py
│   │   ├── litecoreio.py
│   │   ├── multiexplorer.py
│   │   └── services.py
│   ├── tools
│   │   ├── __init__.py
│   │   ├── cli_wallet.py
│   │   ├── mnemonic_key_create.py
│   │   ├── sign_raw.py
│   │   ├── sign_raw_mnemonic.py
│   │   ├── wallet_multisig_2of3.py
│   │   └── wallet_multisig_3of5.py
│   ├── transactions.py
│   ├── wallets.py
│   └── wordlist
│       ├── chinese_simplified.txt
│       ├── chinese_traditional.txt
│       ├── dutch.txt
│       ├── english.txt
│       ├── french.txt
│       ├── italian.txt
│       ├── japanese.txt
│       └── spanish.txt
├── bitcoinlib.egg-info
│   ├── PKG-INFO
│   ├── SOURCES.txt
│   ├── dependency_links.txt
│   ├── entry_points.txt
│   ├── not-zip-safe
│   ├── requires.txt
│   └── top_level.txt
├── docs
│   ├── Makefile
│   ├── _static
│   │   ├── classes-overview-detailed.jpg
│   │   ├── classes-overview-detailed.odt
│   │   ├── classes-overview.jpg
│   │   ├── classes-overview.odt
│   │   ├── default.css
│   │   ├── manuals.add-provider.rst
│   │   ├── manuals.command-line-wallet.rst
│   │   ├── manuals.install.rst
│   │   ├── manuals.setup-bitcoind-connection.rst
│   │   └── script-types-overview.rst
│   ├── _templates
│   │   └── layout.html
│   ├── conf.py
│   ├── index.rst
│   └── requirements.txt
├── examples
│   ├── encoding.py
│   ├── keys.py
│   ├── mnemonic.py
│   ├── networks.py
│   ├── services.py
│   ├── transactions.py
│   ├── transactions_bitcoind_client.py
│   ├── transactions_decompose_simple.py
│   ├── wallets.py
│   ├── wallets_mnemonic.py
│   ├── wallets_multisig.py
│   └── wallets_transactions.py
├── setup.cfg
├── setup.py
├── tests
│   ├── __init__.py
│   ├── bip38_protected_key_tests.json
│   ├── electrum_keys.json
│   ├── mnemonics_tests.json
│   ├── test_custom.py
│   ├── test_encoding.py
│   ├── test_keys.py
│   ├── test_mnemonic.py
│   ├── test_networks.py
│   ├── test_services.py
│   ├── test_tools.py
│   ├── test_transactions.py
│   ├── test_wallets.py
│   └── transactions_raw.json
└── updatedb.py

[arshbot]

Nevermind, found it at ~/.bitcoinlib

[arshbot]

Works like a charm. Thanks @mccwdev

Example

>>> import bitcoinlib
>>> from bitcoinlib.transactions import *
>>> t = Transaction()
>>> prev_tx = 'f2b3eb2deb76566e7324307cd47c35eeb88413f971d88519859b1834307ecfec'
>>> t.add_input(prev_hash=prev_tx, output_n=1, compressed=False)
0

>>> ki = Key(0x18E14A7B6A307F426A94F8114701E7C8E774E7F9A47E2C2035DB29A206321725, compressed=False)
>>> t.add_output(99900000, '1runeksijzfVxyrpiyCY2LCBvYsSiFsCm')
0

>>> t.sign(ki.private_byte)
1

[arshbot]

Another question, how can I get or calculate the txid? The hash attribute is an empty string even after signing.

I know I can just sha256 the raw twice but wondering if there's a way within the library

[mccwdev]

Good point, the hash is only calculated when creating a transaction or by the HDWallet class. But not when signing/updating the transaction or by calling a method. This is probably easy to fix, I put it on my list...