Missing Witness Data and Key Data On PSBT

[itsMikeLowrey]

I am running electrum from the executable in testnet mode and making request over rpc. I used to be able to create a transaction and get a unsigned psbt from the electrum that had bip_32 path data and witness data since I used a vpub. Recently the psbt's send back from electrum have been just having non_witness_utxo data for the input and missing any witness data and key derivation path data. I tried running older versions of electrum(4.0.0b0) and the issue still persist. The wallet still works when I handle all the signing from the gui, but If i do the signing and other stuff of the gui I would be missing a lot of data, that used to be there. I can also post the vpub in question if it would help. I know there have been some changes around witness and utxo data, but Idk if they are at all related to the change that I am experiencing. I would appreciate any help and thank you so much.

[SomberNight]

Could you please detail which RPC commands you use in what order? An example PSBT you get would also be nice.

[itsMikeLowrey]

I was able to recreate this issue through the gui command line. I recoverd a wallet from this vpub: vpub5Y3LsqvBH22zW2oF1V1bTHk47Br3vbETwpNMFX7xgBe9LptUayqwpGYwwBBm23GLUp3otGM5tECqy844sPWiUBTJVckTNtXyySTu18cFHVy . I had already sent some coins to this wallet. I then go to the gui console. version() returns "4.0.4". I then type in: payto("tb1qm5tfegjevj27yvvna9elym9lnzcf0zraxgl8z2", "!", unsigned=True) which returns this pbst: "cHNidP8BAFICAAAAAb+WnT16PfXuOW5xik1YNgMJo0uhui8t2tnwI04LIAOCAAAAAAD9////AZIOAQAAAAAAFgAU3RacollkleIxk+lz8my/mLCXiH3obxwAAAEA/egBAgAAAAABA7jQHOtpvtA9sb/Znz7X6eg3PucNYUBYZbnYvnYgNipHAAAAAAD9////4dnLsJyC+RaBLjepvKz2m8PsimypuvkRWOC79svSBMMAAAAAAP3////996L6yACtEDyX7OebOF4Z95PXf5f8pdBc9nZz/piz8gAAAAAA/f///wE3DwEAAAAAABYAFH0TrtTd5i87gtP/mAE0SsqCbEhPAkcwRAIgOsn2jkTqpwWRFwl4x70n9R2v+pI0izi+bdprYK0AH1gCIEOhHWgWUSIovV2dgaYZLAjhv/JZqI2QWS/Q2k9j0DHEASEDeFtLDAv0jsHPHBO/i3JBADyxzwEmOqWb04+fbLe9qzgCSDBFAiEApyoI3TgLCDQe5dj8gpxAAZYUrhGHOv1slE4RFE8Z5dsCIHDJUQOaIKznmfomxpSbumg8/fvRo+2p3mw+LQoFnML7ASEDbcGB21wH38VeUZoGnOOYRoRmwhQip78DtFOpC8K9egUCRzBEAiBuN0+hNgRcVzJDePjoLvoeemPXtEd9/BJFdkgCQxg/+QIgc5+yUu4PwKIInba2Ye3GqGcmwVWUgXrEyYkFuYxYeUABIQN4W0sMC/SOwc8cE7+LckEAPLHPASY6pZvTj59st72rOOdvHAAiBgKn4O5t4J8XaubCP6ruBTSc1HkYmNZhuVclRe7TFtcjfwytXPTrAAAAAAsAAAAAAA==" I plug this psbt into deserialize() and it gives me: { "inputs": [ { "address": "tb1q05f6a4xauchnhqknl7vqzdz2e2pxcjz08av7r9", "bip32_paths": { "02a7e0ee6de09f176ae6c23faaee05349cd4791898d661b9572545eed316d7237f": [ "ad5cf4eb", "m/0/11" ] }, "coinbase": false, "height": null, "nsequence": 4294967293, "part_sigs": {}, "prevout_hash": "8203200b4e23f0d9da2d2fbaa14ba3090336584d8a716e39eef53d7a3d9d96bf", "prevout_n": 0, "redeem_script": null, "sighash": null, "unknown_psbt_fields": {}, "utxo": "02000000000103b8d01ceb69bed03db1bfd99f3ed7e9e8373ee70d61405865b9d8be7620362a470000000000fdffffffe1d9cbb09c82f916812e37a9bcacf69bc3ec8a6ca9baf91158e0bbf6cbd204c30000000000fdfffffffdf7a2fac800ad103c97ece79b385e19f793d77f97fca5d05cf67673fe98b3f20000000000fdffffff01370f0100000000001600147d13aed4dde62f3b82d3ff9801344aca826c484f0247304402203ac9f68e44eaa70591170978c7bd27f51daffa92348b38be6dda6b60ad001f58022043a11d6816512228bd5d9d81a6192c08e1bff259a88d90592fd0da4f63d031c4012103785b4b0c0bf48ec1cf1c13bf8b7241003cb1cf01263aa59bd38f9f6cb7bdab3802483045022100a72a08dd380b08341ee5d8fc829c40019614ae11873afd6c944e11144f19e5db022070c951039a20ace799fa26c6949bba683cfdfbd1a3eda9de6c3e2d0a059cc2fb0121036dc181db5c07dfc55e519a069ce398468466c21422a7bf03b453a90bc2bd7a050247304402206e374fa136045c57324378f8e82efa1e7a63d7b4477dfc124576480243183ff90220739fb252ee0fc0a2089db6b661edc6a86726c15594817ac4c98905b98c587940012103785b4b0c0bf48ec1cf1c13bf8b7241003cb1cf01263aa59bd38f9f6cb7bdab38e76f1c00", "value_sats": 69431, "witness_script": null, "witness_utxo": null } ], "locktime": 1863656, "outputs": [ { "address": "tb1qm5tfegjevj27yvvna9elym9lnzcf0zraxgl8z2", "bip32_paths": {}, "redeem_script": null, "scriptpubkey": "0014dd169ca2596495e23193e973f26cbf98b097887d", "unknown_psbt_fields": {}, "value_sats": 69266, "witness_script": null } ], "unknown_psbt_fields": {}, "version": 2, "xpubs": {} }

This does have the correct path information and I also checked with the rpc commands that I ran earlier and that is also correct. The only issue that is left is the missing witness data, such as witness_script and witness_utxo. In the past(around 4.0.4b) the psbt that electrum gave me had the witness script for sure, idk about the witness_utxo though. Thanks for the quick response and I hope this was enough info.

[SomberNight]

There is no witness_script because that is for p2wsh scripts; p2wpkh does not have them.

witness_utxo can be derived from utxo, so it is redundant.

Before 4.0.0, for some months on git master, Electrum used to set witness_utxo instead of utxo for segwit inputs (as that is what bip174 specified at the time); but that turned out to be not always enough to safely sign offline (CVE-2020-14199, see https://github.com/spesmilo/electrum/pull/6198), so it was changed to set utxo instead. Some programs, such as Bitcoin Core, decided to set both fields for backwards compatibility, after the CVE was publicised, but we opted to just do a breaking change instead and keep the PSBT smaller.

Does this answer your questions? If so, please close the issue.

[itsMikeLowrey]

Yes. That answers my questions perfectly. Thank you so much for explaining it so well.