cffls
commented
2 years ago Thanks for reporting the issue! It seems to be a bug in UTxO selector. Will look into this.
Closed 34r7h closed 2 years ago
cffls
commented
2 years ago Thanks for reporting the issue! It seems to be a bug in UTxO selector. Will look into this.
cffls
commented
2 years ago Hi @34r7h , the bug should be fixed under this commit: https://github.com/cffls/pycardano/commit/e00b5697b3a00d0110b732048cd9bfa111dd119d Please checkout the latest code and let me know if it works. Thanks!
34r7h
commented
2 years ago Hey @cffls, thanks for your swiftness.
I think you solved an unrelated issue.. this bug seems to come when the coin selection doesn't have a 0 indexed input to choose from.
I checked that the version is 0.4.1. Notice the first transaction log has 2 utxos available:
utxos [{'input': {'index': 1,
'transaction_id': TransactionId(hex='9d255cdacd8a575ee86f4ad0a61b14c7be037c623059f71b1bc9ce8d4e53cb6c')},
'output': {'address': addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d,
'amount': 2821804,
'datum_hash': None}}, {'input': {'index': 0,
'transaction_id': TransactionId(hex='9c9c7ce1ed9018b31c8a3e94475e7f607d87be914091e40d6b44efc711146811')},
'output': {'address': addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d,
'amount': 1000000,
'datum_hash': None}}]While the following transaction has only one utxo (with index: 1) fails:
utxos [{'input': {'index': 1,
'transaction_id': TransactionId(hex='81a471965bfdaf4bc570f7a987c9dbe1bbf01a08564de4c56319746394b5839e')},
'output': {'address': addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d,
'amount': 2652255,
'datum_hash': None}}]
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pycardano/txbuilder.py", line 658, in build
selected, _ = selector.select(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pycardano/coinselection.py", line 109, in select
additional, _ = self.select(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pycardano/coinselection.py", line 94, in select
raise InsufficientUTxOBalanceException("UTxO Balance insufficient!")
pycardano.exception.InsufficientUTxOBalanceException: UTxO Balance insufficient!
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/347rh/a/cardano-python-js/python/createtx.py", line 39, in <module>
signed_tx = builder.build_and_sign([sk], change_address=address)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pycardano/txbuilder.py", line 767, in build_and_sign
tx_body = self.build(change_address=change_address)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pycardano/txbuilder.py", line 690, in build
raise UTxOSelectionException(
pycardano.exception.UTxOSelectionException: All UTxO selectors failed.
Requested output:
{'coin': 1158901, 'multi_asset': {}}
Pre-selected inputs:
{'coin': 0, 'multi_asset': {}}
Additional UTxO pool:
[{'input': {'index': 1,
'transaction_id': TransactionId(hex='81a471965bfdaf4bc570f7a987c9dbe1bbf01a08564de4c56319746394b5839e')},
'output': {'address': addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d,
'amount': 2652255,
'datum_hash': None}}]
Unfulfilled amount:
{'coin': -1493354, 'multi_asset': {}}As a simple hotfix, I'm checking if there's only one utxo and adding it manually. The rest of your logic seems happily intact to complete the tx with proper plumbing for change, etc. <3
if len(utxos) == 1:
print('Only one UTxO. adding raw input..')
builder.add_input(utxos[0])HI @34r7h , I couldn't reproduce the issue after the fix I mentioned above. Here is the code I am running:
network = Network.MAINNET
context = BlockFrostChainContext("mainnetqEZ4wDDoRdtWqh2SNVLNqfQbhlNmTbza", network)
address = Address.from_primitive('addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d')
builder = TransactionBuilder(context)
builder.add_input_address(address)
utxos = context.utxos(str(address))
builder.add_output(
TransactionOutput(
Address.from_primitive(
"addr1qyady0evsaxqsfmz0z8rvmq62fmuas5w8n4m8z6qcm4wrt3e8dlsen8n464ucw69acfgdxgguscgfl5we3rwts4s57ashysyee"
),
Value.from_primitive(
[
1000000,
]
),
)
)
signed_tx = builder.build(change_address=address)It finished successfully without exception. The only difference was that I used build instead of build_and_sign, but the underlying selection logic should be exactly the same. Could you please try again with the latest code of pycardano? Thank you!
34r7h
commented
2 years ago Hey Jer sorry for the delay. I'm still getting the error but using the conditional workaround:
if len(utxos) == 1:
print('add raw input')
builder.add_input(utxos[0])When you tried to reproduce, were you successful with an address containing only 1 utxo?
cffls
commented
2 years ago Hi @34r7h , the address addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d has two utxos, so I modified the code to make sure the chain context only return one, pasted below. You can see that the builder was not called with add_input but still able to build the transaction with only 1 utxo input.
from dataclasses import dataclass, field
from typing import Dict, List
from pycardano import *
network = Network.MAINNET
context = BlockFrostChainContext("mainnetqEZ4wDDoRdtWqh2SNVLNqfQbhlNmTbza", network)
address = Address.from_primitive('addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d')
builder = TransactionBuilder(context)
builder.add_input_address(address)
utxos = context.utxos(str(address))
# Force chain context to return only one utxo
context.utxos = lambda _: utxos[-1:]
builder.add_output(
TransactionOutput(
Address.from_primitive(
"addr1qyady0evsaxqsfmz0z8rvmq62fmuas5w8n4m8z6qcm4wrt3e8dlsen8n464ucw69acfgdxgguscgfl5we3rwts4s57ashysyee"
),
Value.from_primitive(
[
1000000,
]
),
)
)
tx = builder.build(change_address=address)
print(tx.inputs)
print(tx.outputs)Output:
[{'index': 1,
'transaction_id': TransactionId(hex='e6e9ab73f95939c04be8f4f07af9eac7028a12a9f6b1fc2f5dc19509f543da23')}]
[{'address': addr1qyady0evsaxqsfmz0z8rvmq62fmuas5w8n4m8z6qcm4wrt3e8dlsen8n464ucw69acfgdxgguscgfl5we3rwts4s57ashysyee,
'amount': {'coin': 1000000, 'multi_asset': {}},
'datum_hash': None}, {'address': addr1qytqt3v9ej3kzefxcy8f59h9atf2knracnj5snkgtaea6p4r8g3mu652945v3gldw7v88dn5lrfudx0un540ak9qt2kqhfjl0d,
'amount': {'coin': 37118232, 'multi_asset': {}},
'datum_hash': None}]Hi Jerry, not quite the issue brother.. this bug happens if and only if there's one transaction to choose from. The address above has multiple txs already so we can't reproduce.
In the case of a fresh addr, the context.utxos = lambda _: utxos[-1:] results in
raise UTxOSelectionException(
0|crypto | pycardano.exception.UTxOSelectionException: All UTxO selectors failed.
0|crypto | Requested output:
0|crypto | {'coin': 1189528, 'multi_asset': {}}
0|crypto | Pre-selected inputs:
0|crypto | {'coin': 0, 'multi_asset': {}}
0|crypto | Additional UTxO pool:
0|crypto | [] So for some reason, the builder isn't selecting the single transaction as a valid utxo. I'm still using builder.add_input(utxos[0]) if len(utxos) == 1 and that's working fine in this case. I suspect the context return from BF is causing this
Can you try to reproduce again with a fresh address or lmk if there's a mistake I'm missing in this code?
from pycardano import *
import json
import sys
from dataclasses import dataclass, field
from typing import Dict, List
args = sys.argv[1:]
secret = args[0]
data = args[1]
bf = args[2]
dev = False
jsonsecret = json.loads(secret)
jsondata = json.loads(data)
pkey = jsonsecret["payment"]["signing"]["cborHex"]
vkey = jsonsecret["payment"]["verification"]["cborHex"]
network = Network.MAINNET
context = BlockFrostChainContext(bf, network)
sk = PaymentSigningKey.from_cbor(pkey)
vk = PaymentVerificationKey.from_signing_key(sk)
address = Address.from_primitive(jsondata["address"])
builder = TransactionBuilder(context)
utxos = context.utxos(str(address))
if len(utxos) == 1:
builder.add_input(utxos[0])
# context.utxos = lambda _: utxos[-1:]
else:
builder.add_input_address(address)
for x in jsondata["outputs"]:
outputaddress = x["address"]
tokens = [2000000]
for y in x["tokens"]:
if y["unit"] == "lovelace":
tokens[0] = int(y["quantity"])
else:
policyid = y["unit"][0 : 56]
tokenname = y["unit"][-30:len(y["unit"])]
tokens.append(
{
bytes.fromhex(policyid): {
bytes.fromhex(tokenname): int(y["quantity"]) # Asset name and amount
}
}
)
builder.add_output(
TransactionOutput(
Address.from_primitive(outputaddress), Value.from_primitive(tokens)
)
)
signed_tx = builder.build_and_sign([sk], change_address=address)
tx_id = str(signed_tx.id)
context.submit_tx(signed_tx.to_cbor())
Hi @34r7h , I didn't mean to let you to put context.utxos = lambda _: utxos[-1:] in your production code. I was using it simply to demonstrate that the bug has been fixed. Because the example I posted contains more than one utxo, I had to fake its utxos so the build will only see one in the list.
Previously, I meant your code will still work fine with this simplification (please make sure the version of pycardano has been upgraded to v0.5.0):
changing this:
if len(utxos) == 1:
builder.add_input(utxos[0])
# context.utxos = lambda _: utxos[-1:]
else:
builder.add_input_address(address)to this:
builder.add_input_address(address)If you have an address that contains only one utxo, I am happy to test it out for you.
34r7h
commented
2 years ago Ah, ye this issue is when only one tx exists on an address. Here's an address with only one: addr1qypm6f2z5g45duzj9v9lt7jz9ce2q5m59vw3reqm9e25uxpynes82004nuvufjx0zu8up9dlr574azfnnp2vj3dcwrsqfux5t0
cffls
commented
2 years ago It is working correctly with the address you provided:
network = Network.MAINNET
context = BlockFrostChainContext("mainnetqEZ4wDDoRdtWqh2SNVLNqfQbhlNmTbza", network)
address = Address.from_primitive('addr1qypm6f2z5g45duzj9v9lt7jz9ce2q5m59vw3reqm9e25uxpynes82004nuvufjx0zu8up9dlr574azfnnp2vj3dcwrsqfux5t0')
builder = TransactionBuilder(context)
builder.add_input_address(address)
builder.add_output(
TransactionOutput(
Address.from_primitive(
"addr1qyady0evsaxqsfmz0z8rvmq62fmuas5w8n4m8z6qcm4wrt3e8dlsen8n464ucw69acfgdxgguscgfl5we3rwts4s57ashysyee"
),
Value.from_primitive(
[
1000000,
]
),
)
)
tx_body = builder.build(change_address=address)
print(tx_body)Output:
{'auxiliary_data_hash': None,
'certificates': None,
'collateral': None,
'collateral_return': None,
'fee': 167965,
'inputs': [{'index': 0,
'transaction_id': TransactionId(hex='1764ea2ce4653c1f03b78a5ac73cf4c40247a930f60d3467927f744e9c06fc6d')}],
'mint': None,
'network_id': None,
'outputs': [{'address': addr1qyady0evsaxqsfmz0z8rvmq62fmuas5w8n4m8z6qcm4wrt3e8dlsen8n464ucw69acfgdxgguscgfl5we3rwts4s57ashysyee,
'amount': {'coin': 1000000, 'multi_asset': {}},
'datum_hash': None},
{'address': addr1qypm6f2z5g45duzj9v9lt7jz9ce2q5m59vw3reqm9e25uxpynes82004nuvufjx0zu8up9dlr574azfnnp2vj3dcwrsqfux5t0,
'amount': {'coin': 1662574, 'multi_asset': {}},
'datum_hash': None}],
'reference_inputs': None,
'required_signers': None,
'script_data_hash': None,
'total_collateral': None,
'ttl': None,
'update': None,
'validity_start': None,
'withdraws': None}Cheers for continuing to humor me on this.. yes, build works for me too but build_and_sign throws the error. Could there be a hiccup in the extra fee for the sig?
I'm fine with my conditional check, so nothing is breaking and all good. At your convenience, plz check build_and_sign from an address you can sign. To reproduce, it must have only one tx
cffls
commented
2 years ago Hi @34r7h , I find it hard to believe build works but build_and_sign throws error. What error did you see?
I created an address that has only one UTxO in it and ran the following code. Everything is working correctly.
Code:
from blockfrost import BlockFrostApi, ApiUrls
from pycardano import *
network = Network.TESTNET
PAYMENT_KEY_PATH = "payment2.skey"
context = BlockFrostChainContext("my_testnet_project_id", network)
psk = PaymentSigningKey.load(PAYMENT_KEY_PATH)
pvk = PaymentVerificationKey.from_signing_key(psk)
address = Address(pvk.hash(), network=network)
print("Address: ", address)
print("UTxOs: ", context.utxos(str(address)))
builder = TransactionBuilder(context)
builder.add_input_address(address)
builder.add_output(
TransactionOutput(
Address.from_primitive(
"addr_test1vzvj0223pmnnyyjkcqwnpt0rszlk5rtpdp3d7necq60wzmgt4h5rr"
),
Value.from_primitive(
[
1000000,
]
),
)
)
tx = builder.build_and_sign(change_address=address, signing_keys=[psk])
print("Transaction: ", tx)Output:
Address: addr_test1vqdfs0vy0eraw5xcpj89qkp0v38v2g6xkzzajp33tzs04vq8hq40k
UTxOs: [{'input': {'index': 0,
'transaction_id': TransactionId(hex='e22ea2413a8416b8169a3c8a5180c9a54cd8a463eaaf49a527812e4bdd7bcc8e')},
'output': {'address': addr_test1vqdfs0vy0eraw5xcpj89qkp0v38v2g6xkzzajp33tzs04vq8hq40k,
'amount': {'coin': 2800000, 'multi_asset': {}},
'datum_hash': None}}]
Transaction: {'auxiliary_data': None,
'transaction_body': {'auxiliary_data_hash': None,
'certificates': None,
'collateral': None,
'collateral_return': None,
'fee': 165413,
'inputs': [{'index': 0,
'transaction_id': TransactionId(hex='e22ea2413a8416b8169a3c8a5180c9a54cd8a463eaaf49a527812e4bdd7bcc8e')}],
'mint': None,
'network_id': None,
'outputs': [{'address': addr_test1vzvj0223pmnnyyjkcqwnpt0rszlk5rtpdp3d7necq60wzmgt4h5rr,
'amount': {'coin': 1000000, 'multi_asset': {}},
'datum_hash': None},
{'address': addr_test1vqdfs0vy0eraw5xcpj89qkp0v38v2g6xkzzajp33tzs04vq8hq40k,
'amount': {'coin': 1634587, 'multi_asset': {}},
'datum_hash': None}],
'reference_inputs': None,
'required_signers': None,
'script_data_hash': None,
'total_collateral': None,
'ttl': None,
'update': None,
'validity_start': None,
'withdraws': None},
'transaction_witness_set': {'bootstrap_witness': None,
'native_scripts': None,
'plutus_data': None,
'plutus_script': None,
'redeemer': None,
'vkey_witnesses': [{'signature': b"\x0f\xb035[\xfc;\xa2\xca\xedbk\x84';h\xfa\xf0=\x92\xc5?\xff\xdb"
b'\x06J\x12\x0b\x86\x80\xc7\x07Tk\xad\xf1\xe9\xf6\xb7?'
b'\xebG\xfb\xec\r\xf9w\x9bg\xa3\xfc\xc8d8\x80#B\x83Q\xd5'
b'g\xa4\xb6\x00',
'vkey': {"type": "PaymentVerificationKeyShelley_ed25519", "description": "PaymentVerificationKeyShelley_ed25519", "cborHex": "58209ce6f79cc94658844a8651607242b6e02388da183dcecfd62389aad676597ac1"}}]},
'valid': True}Ok you're right. I was testing with addresses having less than ~2.4 ada so was failing as expected. I'll close this issue as it's really the same problem you've fixed in a recent commit. Will test and follow-up if same trouble happens later. Thanks for everything!
Hello there, hope all is smooth.
I'm running into an issue with the builder.. seems I need > 1 utxo in order to send ada to an address.
Successful tx has utxo data like this:
Unsuccessful utxo data looks like this:
So, I suspect the problem comes when there's no index[0] in the list.. and I get this UTxOSelectionException error:
My script is essentially the same as your example: