cryptoadvance / specter-desktop

A desktop GUI for Bitcoin Core optimised to work with hardware wallets
MIT License
793 stars 236 forks source link

Failed to load UTXOs with Taproot wallet #2078

Open moneymanolis opened 1 year ago

moneymanolis commented 1 year ago
grafik

(Screenshot from revamp branch but same on master)

Steps to reproduce

Create a Taproot wallet, send regtest funds to it from a Segwit wallet to produce an UTXO for the Taproot wallet.

Stacktrace

Updating wallet Taproot only wallet
[2023-01-25 10:27:02,092] ERROR in wallet: 'fc22c9b4dcb1f4b063852c4fb1e59dd9806c691583c6b5b385cf3c80488ef464'
Traceback (most recent call last):
  File "/Users/heinzi/coding/repos/specter-desktop/src/cryptoadvance/specter/wallet.py", line 845, in check_utxo
    tx: WalletAwareTxItem = txlist_dict[utxo_txid]
KeyError: 'fc22c9b4dcb1f4b063852c4fb1e59dd9806c691583c6b5b385cf3c80488ef464'
Exception in thread Thread-144 (_update):
Traceback (most recent call last):
  File "/Users/heinzi/coding/repos/specter-desktop/src/cryptoadvance/specter/wallet.py", line 845, in check_utxo
    tx: WalletAwareTxItem = txlist_dict[utxo_txid]
KeyError: 'fc22c9b4dcb1f4b063852c4fb1e59dd9806c691583c6b5b385cf3c80488ef464'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/heinzi/.pyenv/versions/3.10.0/lib/python3.10/threading.py", line 1009, in _bootstrap_inner
    self.run()
  File "/Users/heinzi/coding/repos/specter-desktop/src/cryptoadvance/specter/util/flask.py", line 32, in run
    super().run()
  File "/Users/heinzi/.pyenv/versions/3.10.0/lib/python3.10/threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/heinzi/coding/repos/specter-desktop/src/cryptoadvance/specter/managers/wallet_manager.py", line 259, in _update
    self.wallets[wallet_name].update()
  File "/Users/heinzi/coding/repos/specter-desktop/src/cryptoadvance/specter/wallet.py", line 635, in update
    self.getdata()
  File "/Users/heinzi/coding/repos/specter-desktop/src/cryptoadvance/specter/wallet.py", line 929, in getdata
    self.check_utxo()
  File "/Users/heinzi/coding/repos/specter-desktop/src/cryptoadvance/specter/wallet.py", line 864, in check_utxo
    raise SpecterError(f"Failed to load utxos, {e}")
cryptoadvance.specter.specter_error.SpecterError: Failed to load utxos, 'fc22c9b4dcb1f4b063852c4fb1e59dd9806c691583c6b5b385cf3c80488ef464'
k9ert commented 1 year ago

For a strange reason, this is only a problem until specter is rebooted. Then it disappears. At least i know roughly what's the case here: the taproot transaction is not be considered as ismine. This is implemented like this:

    @property
    def ismine(self) -> bool:
        if self.get("ismine"):
            return self["ismine"]
        inputs = self.psbt.inputs
        outputs = self.psbt.outputs
        any_inputs_mine = any([inp.is_mine for inp in inputs])
        any_outputs_mine = any([out.is_mine for out in outputs])
        self["ismine"] = any_inputs_mine or any_outputs_mine
        return self["ismine"]

The psbt is created like this:

    @property
    def psbt(self) -> SpecterPSBT:
        """This tx but as a psbt. Need rpc-calls"""
        if hasattr(self, "_psbt"):
            return self._psbt
        self._psbt: SpecterPSBT = self.PSBTCls.from_transaction(
            self.tx, self.descriptor, self.network
        )
        # fill derivation paths etc
        updated = self.rpc.walletprocesspsbt(str(self._psbt), False).get("psbt", None)
        if updated:
            self._psbt.update(updated)
        return self._psbt

As this needs to be an issue for the output as the output needs to be ismine, the SpecterOutputScope.ismine implementation:

    @property
    def is_mine(self) -> bool:
        return self.descriptor.owns(self.scope)

@stepansnigirev can you see any reason why taproot transactions have trouble here?

k9ert commented 1 year ago

I've figured out that the cache has the right values but the implementations which are SUPPOSE to calculate the cached value does not: image

So there is somewhere some taproot magic which i'm not aware of.

stepansnigirev commented 1 year ago

If Bitcoin Core does not add taproot fields into PSBT, the descriptor can't verify that the output belongs to the descriptor. What version of Core do you use?

stepansnigirev commented 1 year ago

Oh wait... Specter is creating psbt from transaction itself, then probably the problem is with taproot fields that we don't add in the code.

k9ert commented 1 year ago

I'm using:

  "version": 220000,
  "subversion": "/Satoshi:22.0.0/",

Here is the psbt from that tx.

Click to expand ``` cHNidP8BAH0CAAAAAeU5AjQ3OkQNmC7Act0zvmrRa98XZaH73MV3QWzFbWc0AAAAAAD+////AgQlEwAAAAAAFgAUDshyhY0ZMuW1aDjZW01Pc7Mz63qAlpgAAAAAACJRIGhPDG28WaL+B0Dxql0du4vf2RfEixwwM+Vi4T+8ecKQZQoAAAABAH0CAAAAAcMXj0XQ/cNHEUxb0dd9aKRoovQHvFHEj2gwXLhxvLlAAAAAAAD+////AnjHqwAAAAAAFgAUFuYflFUX6IXeEKF/lNf1GqZx9J6AlpgAAAAAACJRIMKyPPj50EhToC/taMfusSreBpZE8XwrVbMk2Zibm82IMwoAAAEBH3jHqwAAAAAAFgAUFuYflFUX6IXeEKF/lNf1GqZx9J4AAAA= ``` ```bitcoin-cli -regtest -rpcport=18443 -rpcuser=bitcoin -rpcpassword=secret decodepsbt "cHNidP8BAH0CAAAAAeU5AjQ3OkQNmC7Act0zvmrRa98XZaH73MV3QWzFbWc0AAAAAAD+////AgQlEwAAAAAAFgAUDshyhY0ZMuW1aDjZW01Pc7Mz63qAlpgAAAAAACJRIGhPDG28WaL+B0Dxql0du4vf2RfEixwwM+Vi4T+8ecKQZQoAAAABAH0CAAAAAcMXj0XQ/cNHEUxb0dd9aKRoovQHvFHEj2gwXLhxvLlAAAAAAAD+////AnjHqwAAAAAAFgAUFuYflFUX6IXeEKF/lNf1GqZx9J6AlpgAAAAAACJRIMKyPPj50EhToC/taMfusSreBpZE8XwrVbMk2Zibm82IMwoAAAEBH3jHqwAAAAAAFgAUFuYflFUX6IXeEKF/lNf1GqZx9J4AAAA=" { "tx": { "txid": "fdd7f2fee8a9e7a1c641898b07cab90df7adad6d698947d19e165f2ad4f22805", "hash": "fdd7f2fee8a9e7a1c641898b07cab90df7adad6d698947d19e165f2ad4f22805", "version": 2, "size": 125, "vsize": 125, "weight": 500, "locktime": 2661, "vin": [ { "txid": "34676dc56c4177c5dcfba16517df6bd16abe33dd72c02e980d443a37340239e5", "vout": 0, "scriptSig": { "asm": "", "hex": "" }, "sequence": 4294967294 } ], "vout": [ { "value": 0.01254660, "n": 0, "scriptPubKey": { "asm": "0 0ec872858d1932e5b56838d95b4d4f73b333eb7a", "hex": "00140ec872858d1932e5b56838d95b4d4f73b333eb7a", "address": "bcrt1qpmy89pvdryewtdtg8rv4kn20wwen86m6jpflrx", "type": "witness_v0_keyhash" } }, { "value": 0.10000000, "n": 1, "scriptPubKey": { "asm": "1 684f0c6dbc59a2fe0740f1aa5d1dbb8bdfd917c48b1c3033e562e13fbc79c290", "hex": "5120684f0c6dbc59a2fe0740f1aa5d1dbb8bdfd917c48b1c3033e562e13fbc79c290", "address": "bcrt1pdp8scmdutx30up6q7x4968dm300aj97y3vwrqvl9vtsnl0rec2gq0kytph", "type": "witness_v1_taproot" } } ] }, "unknown": { }, "inputs": [ { "witness_utxo": { "amount": 0.11257720, "scriptPubKey": { "asm": "0 16e61f945517e885de10a17f94d7f51aa671f49e", "hex": "001416e61f945517e885de10a17f94d7f51aa671f49e", "type": "witness_v0_keyhash", "address": "bcrt1qzmnpl9z4zl5gthss59lef4l4r2n8ray7m8ln0a" } }, "non_witness_utxo": { "txid": "34676dc56c4177c5dcfba16517df6bd16abe33dd72c02e980d443a37340239e5", "hash": "34676dc56c4177c5dcfba16517df6bd16abe33dd72c02e980d443a37340239e5", "version": 2, "size": 125, "vsize": 125, "weight": 500, "locktime": 2611, "vin": [ { "txid": "40b9bc71b85c30688fc451bc07f4a268a4687dd7d15b4c1147c3fdd0458f17c3", "vout": 0, "scriptSig": { "asm": "", "hex": "" }, "sequence": 4294967294 } ], "vout": [ { "value": 0.11257720, "n": 0, "scriptPubKey": { "asm": "0 16e61f945517e885de10a17f94d7f51aa671f49e", "hex": "001416e61f945517e885de10a17f94d7f51aa671f49e", "address": "bcrt1qzmnpl9z4zl5gthss59lef4l4r2n8ray7m8ln0a", "type": "witness_v0_keyhash" } }, { "value": 0.10000000, "n": 1, "scriptPubKey": { "asm": "1 c2b23cf8f9d04853a02fed68c7eeb12ade069644f17c2b55b324d9989b9bcd88", "hex": "5120c2b23cf8f9d04853a02fed68c7eeb12ade069644f17c2b55b324d9989b9bcd88", "address": "bcrt1pc2ere78e6py98gp0a45v0m439t0qd9jy797zk4dnynve3xumekyqc3qsjw", "type": "witness_v1_taproot" } } ] } } ], "outputs": [ { }, { } ], "fee": 0.00003060 } ```
stepansnigirev commented 1 year ago

Yes, as you see, inputs and outputs don't contain derivation information, so descriptor.owns() always returns False. With non-taproot wallets walletprocesspsbt rpc call populates PSBT with derivation information, but not for taproot: https://github.com/cryptoadvance/specter-desktop/blob/fa9605d6c955f46bce441c759f6bf799a8c74ab3/src/cryptoadvance/specter/txlist.py#L321

It could be that Bitcoin Core 23 does it, but I didn't check

k9ert commented 1 year ago

But then, how does the right information gets into the cache? My gues is that the super-confusing fetch_transaction-method does that in this section:

Click to expand ```python if self.use_descriptors: # Get all used addresses that belong to the wallet addresses_info_multi = self.rpc.multi( [ ("getaddressinfo", address) for address in [ tx["details"][0].get("address") for tx in txs.values() if tx and tx.get("details") and ( tx.get("details")[0].get("category") != "send" and tx["details"][0].get("address") not in self._addresses ) ] if address ] ) addresses_info = [ r["result"] for r in addresses_info_multi if r["result"].get("ismine", False) ] ```

Update: That assumption was wrong.

stepansnigirev commented 1 year ago

Maybe wallet.fill_psbt can help to get full information. https://github.com/cryptoadvance/specter-desktop/blob/ff94b0651123426843786d2a3f07bcc2392673a0/src/cryptoadvance/specter/wallet.py#L1943

stepansnigirev commented 1 year ago

Yes, another way is to check if addresses used in inputs and outputs belong to the wallet.

k9ert commented 1 year ago

It's an amazing feature. I'll call that from now on an Heisenberg Feature.

Update:Ok, i have the "feature". The value in the csv was "false". And we have a type_converter which is bool, but:

>>> bool("False")
True
>>> 
k9ert commented 1 year ago

The result of fill_psbt would look like this:

Click to expand ``` bitcoin-cli -regtest -rpcport=18443 -rpcuser=bitcoin -rpcpassword=secret -rpcwallet="spectera8d0c3bcfa1dd45c/mytaproot" decodepsbt cHNidP8BAH0CAAAAAeU5AjQ3OkQNmC7Act0zvmrRa98XZaH73MV3QWzFbWc0AAAAAAD+////AgQlEwAAAAAAFgAUDshyhY0ZMuW1aDjZW01Pc7Mz63qAlpgAAAAAACJRIGhPDG28WaL+B0Dxql0du4vf2RfEixwwM+Vi4T+8ecKQZQoAAAABAH0CAAAAAcMXj0XQ/cNHEUxb0dd9aKRoovQHvFHEj2gwXLhxvLlAAAAAAAD+////AnjHqwAAAAAAFgAUFuYflFUX6IXeEKF/lNf1GqZx9J6AlpgAAAAAACJRIMKyPPj50EhToC/taMfusSreBpZE8XwrVbMk2Zibm82IMwoAAAEBH3jHqwAAAAAAFgAUFuYflFUX6IXeEKF/lNf1GqZx9J4AAAEFIKecIuQuHcy/Qk9WCw7fpl65aZtsV4cbIyCEhncl+5XXIQennCLkLh3Mv0JPVgsO36ZeuWmbbFeHGyMghIZ3JfuV1xkAxu/5fVYAAIABAACAAAAAgAAAAAACAAAAAA== { "tx": { "txid": "fdd7f2fee8a9e7a1c641898b07cab90df7adad6d698947d19e165f2ad4f22805", "hash": "fdd7f2fee8a9e7a1c641898b07cab90df7adad6d698947d19e165f2ad4f22805", "version": 2, "size": 125, "vsize": 125, "weight": 500, "locktime": 2661, "vin": [ { "txid": "34676dc56c4177c5dcfba16517df6bd16abe33dd72c02e980d443a37340239e5", "vout": 0, "scriptSig": { "asm": "", "hex": "" }, "sequence": 4294967294 } ], "vout": [ { "value": 0.01254660, "n": 0, "scriptPubKey": { "asm": "0 0ec872858d1932e5b56838d95b4d4f73b333eb7a", "hex": "00140ec872858d1932e5b56838d95b4d4f73b333eb7a", "address": "bcrt1qpmy89pvdryewtdtg8rv4kn20wwen86m6jpflrx", "type": "witness_v0_keyhash" } }, { "value": 0.10000000, "n": 1, "scriptPubKey": { "asm": "1 684f0c6dbc59a2fe0740f1aa5d1dbb8bdfd917c48b1c3033e562e13fbc79c290", "hex": "5120684f0c6dbc59a2fe0740f1aa5d1dbb8bdfd917c48b1c3033e562e13fbc79c290", "address": "bcrt1pdp8scmdutx30up6q7x4968dm300aj97y3vwrqvl9vtsnl0rec2gq0kytph", "type": "witness_v1_taproot" } } ] }, "unknown": { }, "inputs": [ { "witness_utxo": { "amount": 0.11257720, "scriptPubKey": { "asm": "0 16e61f945517e885de10a17f94d7f51aa671f49e", "hex": "001416e61f945517e885de10a17f94d7f51aa671f49e", "type": "witness_v0_keyhash", "address": "bcrt1qzmnpl9z4zl5gthss59lef4l4r2n8ray7m8ln0a" } }, "non_witness_utxo": { "txid": "34676dc56c4177c5dcfba16517df6bd16abe33dd72c02e980d443a37340239e5", "hash": "34676dc56c4177c5dcfba16517df6bd16abe33dd72c02e980d443a37340239e5", "version": 2, "size": 125, "vsize": 125, "weight": 500, "locktime": 2611, "vin": [ { "txid": "40b9bc71b85c30688fc451bc07f4a268a4687dd7d15b4c1147c3fdd0458f17c3", "vout": 0, "scriptSig": { "asm": "", "hex": "" }, "sequence": 4294967294 } ], "vout": [ { "value": 0.11257720, "n": 0, "scriptPubKey": { "asm": "0 16e61f945517e885de10a17f94d7f51aa671f49e", "hex": "001416e61f945517e885de10a17f94d7f51aa671f49e", "address": "bcrt1qzmnpl9z4zl5gthss59lef4l4r2n8ray7m8ln0a", "type": "witness_v0_keyhash" } }, { "value": 0.10000000, "n": 1, "scriptPubKey": { "asm": "1 c2b23cf8f9d04853a02fed68c7eeb12ade069644f17c2b55b324d9989b9bcd88", "hex": "5120c2b23cf8f9d04853a02fed68c7eeb12ade069644f17c2b55b324d9989b9bcd88", "address": "bcrt1pc2ere78e6py98gp0a45v0m439t0qd9jy797zk4dnynve3xumekyqc3qsjw", "type": "witness_v1_taproot" } } ] } } ], "outputs": [ { }, { "unknown": { "05": "a79c22e42e1dccbf424f560b0edfa65eb9699b6c57871b232084867725fb95d7", "07a79c22e42e1dccbf424f560b0edfa65eb9699b6c57871b232084867725fb95d7": "00c6eff97d5600008001000080000000800000000002000000" } } ], "fee": 0.00003060 } ```

So this looks like exactly the original psbt.

Update: Bitcoin core can't see this fields as that is the point of the fill_psbt method. So decoding it with the help of Core is useless.

stepansnigirev commented 1 year ago

Output contains "unknown" fields that are actually taproot derivations. Bitcoin Core doesn't know what they mean but embit descriptor should be able to detect owned outputs

k9ert commented 1 year ago

A sideeffect of the mitigation of this issue is that the amounts are broken on taproot because is_mine doesn't work properly on the in-/outputs but that's essential to calculate the amounts.