`VersionTransaction.message` doesn't seem to contain the initial message byte, which prevents proper signing

[aspin]

I have an unsigned transaction message with these bytes:

AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAQAHCibUXsV2mzsJIvnE2gwNr+KfdyhHR5jyjkRbbnsGqAqrbpmrfEFlbO4+KjKolmWHzvZZb9rc1UMgak9Lohf1TbIuGahV4LYOAa+/QpNyN+z24Zpt5aOJKfYLZNemgIMEIwMGRm/lIRcy/+ytunLDm+e8jOW7xfcSayxDmzpAAAAAjJclj04kifG7PRApFI4NgwtaE5na/xCEBI572Nvp+FkGm4hX/quBhPtof2NGGMA12sQ53BrrO1WYoPAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABt324ddloZPZy+FGzut5rBy0he1fWzeROoz1hX7/AKnG+nrzvtutOj1l82qryXQxsbvkwtL24OR8pgIDRS9dYQR51S3tv2vF7NCdhFNKNK6ll1BDs2/QKyRlC7WEQ1lcqDfg9KjP9p0wygkxqsEwpYGnVWUr/AMYoE3FpEQfz4YHAwAFAsBcFQAEBgABAAUGBwAGAgABDAIAAACAlpgAAAAAAAcBAQERBAYAAgAIBgcACREHAAIOAA8KAQILDA0OBxANCSLlF8uXeuOtKgABAAAAAguAlpgAAAAAAL3fAwAAAAAAAQAABwMBAAABCQFX3lqkjZImQIEkZIQeRHTRorA35IRt35MRRVgR3QxiLARfYGFiA11eZQ==

    b = base64.b64decode(unsigned_tx_base64) # from snippet above
    raw_tx = solders_tx.VersionedTransaction.from_bytes(b)
    message = raw_tx.message
    signature = keypair.sign_message(message.to_json())
    signatures = [signature]

    if len(raw_tx.signatures) > 1:
        signatures.extend(list(raw_tx.signatures[1:]))

    tx = solders_tx.VersionedTransaction.populate(message, signatures)

But the resulting signature is wrong. I looked a bit deeper at this, and I see that the first byte of raw_tx.message is different from other sources (I was comparing against https://github.com/gagliardetto/solana-go).

> [i for i in  bytes(message)[:10]]

[1, 0, 7, 10, 38, 212, 94, 197, 118, 155]

When I'm comparing against the other library, where the transaction is submitted properly:

[128, 1, 0, 7, 10, 38, 212, 94, 197, 118, ...]

Any suggestions on the correct way to sign the message?

[aspin]

I'm hacking around this right now by manually appending the byte, but the ability to determine if a transaction is legacy is pretty sketchy:

    raw_tx = solders_tx.VersionedTransaction.from_bytes(b)

    version = raw_tx.version()

    if version is solders_tx.Legacy.Legacy: # doesn't work, like at all. have to do str(version) == "Legacy.Legacy"
        message_bytes = bytes(raw_tx.message)
    else:
        padding = 128
        message_bytes = padding.to_bytes(1, 'big') + bytes(raw_tx.message)
    signature = keypair.sign_message(message_bytes)

[kevinheavey]

    if version is solders_tx.Legacy.Legacy: # doesn't work, like at all. have to do str(version) == "Legacy.Legacy"
        message_bytes = bytes(raw_tx.message)

== works btw:

if version == solders_tx.Legacy.Legacy:

Taking a look at the rest now.

[kevinheavey]

Do you have an example where you build the transaction up rather than just reading the base64? That would help rule out a few things

[aspin]

It's a little bit tricky for me to produce the generation code since it's coming through a bunch of other services in other languages. I can tell you that it's a swap being done through Jupiter (https://www.npmjs.com/package/@jup-ag/core), but I'm not sure that's super helpful given that all their code is minified as is. I can see if I can produce another example though.

Problem with version == solders_tx.Legacy.Legacy is that 0 passes that check. So can't distinguish between legacy vs v0 transaction

[aspin]

for the second:

UNSIGNED_TX_1 = "AgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQD6Amzo3TOF19nBfTBPCjx2oV1dRi5AtFA3kxVbfyOai9/PDbvr0d9iYkVVYCo+5heOy6m60Ua60t+EF+kXEJAgAFD4ls26fgpAnCYufUzDrXMMpDjMYkf2Y2FHuxqKE+2+IrRVKOQVHKKvreZyvh3wca8QpEP1VhjdfPmQtxZk41vr2EwvsYrtYZ9UZjJlPvBgKfAqhkvzgphnGBuyDfHXFcMEoHX5xmEhJgK0YZx3BKh/s3nhpE7IFyBzqsKBqiTDd6jfzI9XsPznt1ZnWa9u9nVKg1KibD5ElrzSfbftYpluJAIIlGU8/d+nt+YMlmaCc2otsPg4VkklsRB3oh4DbXlwD0JuFuuM8DEZF1+YBRQ0SVXONw52WUDzwpQ5VF+0Wppt/RXFB3Bfkzm5U8Gk39vJzBht0vYt9IqVgEXip2UlkfJvXwRhxAEL1cyMpwZt2lhKbucXk0xnet9MJfvRVqLWrj7TJ6D4hJp3KUHZcFDzpujLjdOrzbFHCIfIK1TT82BpuIV/6rgYT7aH9jRhjANdrEOdwa6ztVmKDwAAAAAAEGp9UXGSxcUSGMyUw9SvF/WNruCJuh/UTj29mKAAAAAAbd9uHXZaGT2cvhRs7reawctIXtX1s3kTqM9YV+/wCpAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACFDy1uAqR6+CTQmradxC1wyyjL+iSft+5XudJWwSdi7817DQWO+TIR4kugIb5dD6FCxudqoRDfwOC8xhRQk5dqBA0CAAE0AAAAAIB3jgYAAAAApQAAAAAAAAAG3fbh12Whk9nL4UbO63msHLSF7V9bN5E6jPWFfv8AqQwEAQoACwEBDgwCAwQFBgcBAAgJDAszAAoAAAABAAAAwAslCgAAAAABAAAAAAAAAACXePYDAAAAAAAAAAAAAAAAAAAAAAAAAP//DAMBAAABCQ=="

UNSIGNED_TX_2 = "AQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAQAHCibUXsV2mzsJIvnE2gwNr+KfdyhHR5jyjkRbbnsGqAqrbpmrfEFlbO4+KjKolmWHzvZZb9rc1UMgak9Lohf1TbIuGahV4LYOAa+/QpNyN+z24Zpt5aOJKfYLZNemgIMEIwMGRm/lIRcy/+ytunLDm+e8jOW7xfcSayxDmzpAAAAAjJclj04kifG7PRApFI4NgwtaE5na/xCEBI572Nvp+FkGm4hX/quBhPtof2NGGMA12sQ53BrrO1WYoPAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABt324ddloZPZy+FGzut5rBy0he1fWzeROoz1hX7/AKnG+nrzvtutOj1l82qryXQxsbvkwtL24OR8pgIDRS9dYQR51S3tv2vF7NCdhFNKNK6ll1BDs2/QKyRlC7WEQ1lcqDfg9KjP9p0wygkxqsEwpYGnVWUr/AMYoE3FpEQfz4YHAwAFAsBcFQAEBgABAAUGBwAGAgABDAIAAACAlpgAAAAAAAcBAQERBAYAAgAIBgcACREHAAIOAA8KAQILDA0OBxANCSLlF8uXeuOtKgABAAAAAguAlpgAAAAAAL3fAwAAAAAAAQAABwMBAAABCQFX3lqkjZImQIEkZIQeRHTRorA35IRt35MRRVgR3QxiLARfYGFiA11eZQ=="

b1 = base64.b64decode(UNSIGNED_TX_1)
tx1 = solders.transaction.VersionedTransaction.from_bytes(b1)
print(tx1.version()) # Legacy.Legacy
print(tx1.version() == solders.transaction.Legacy.Legacy) # True

b2 = base64.b64decode(UNSIGNED_TX_2)
tx2 = solders.transaction.VersionedTransaction.from_bytes(b2)
print(tx2.version()) # 0
print(tx2.version() == solders.transaction.Legacy.Legacy) # True

[aspin]

Hm, I can't really reproduce with a new transaction. But isn't the base64 provided example fairly straightforward? In the transaction, the first 65 bytes consist of a byte indicating 1 signature, then 64 bytes of an empty signature, then the rest of the message.

In the rest of the message, the first byte indicates the transaction version. 128 indicates v0, <127 indicates legacy type. There's basically no way the rest of the message would affect the parsing of the message in this way, would it?

[aspin]

Actually wait, I have something for you.

import unittest

from solders import hash as hs
from solders import instruction as inst
from solders import keypair as kp
from solders import message as msg
from solders import pubkey as pk
from solders import transaction as tx

# randomly generated key
PRIVATE_KEY = kp.Keypair.from_base58_string(
    "3KWC65p6AvMjvpR2r1qLTC4HVSH4jEFr5TMQxagMLo1o3j4yVYzKsfbB3jKtu3yGEHjx2Cc3L5t8wSo91vpjT63t"
)
PUBLIC_KEY = PRIVATE_KEY.pubkey()
HASH = hs.Hash.from_string("4uQeVj5tqViQh7yWWGStvkEG1Zmhx6uasJtWCJziofM")

class TestExample(unittest.TestCase):
    def test_sign_tx(self):
        programKey = pk.Pubkey.from_string(
            "HQ2UUt18uJqKaQFJhgV9zaTdQxUZjNrsKFgoEDquBkcx"
        )
        instruction = inst.Instruction(programKey, bytes("123", "utf-8"), [])

        original_msg = msg.MessageV0.try_compile(
            PUBLIC_KEY, [instruction], [], HASH
        )

        signed_tx = tx.VersionedTransaction(original_msg, [PRIVATE_KEY])
        print(bytes(signed_tx))
        signed_tx.verify_and_hash_message()  # ok

        signature = PRIVATE_KEY.sign_message(bytes(original_msg))
        signed_tx_populate = tx.VersionedTransaction.populate(
            original_msg, [signature]
        )
        print(bytes(signed_tx_populate))
        signed_tx_populate.verify_and_hash_message()  # fails!

You must do something different in the constructor for VersionedTransaction when you signed the message compared to when getting the attribute directly.

[kevinheavey]

Making a PR so Legacy.Legacy isn't implicitly cast to int when checking equality. In the meantime isinstance(v, Legacy) should work

[kevinheavey]

Thanks for this, might have taken a lot longer to fix without all the extras