Add and stabilize a basic interface for signing transactions

[jsdw]

We currently have a smoldot-v1 interface for communicating with a Smoldot in an extension, and a substrate-connect-unstable interface for talking with extensions like Substrate Connect (this hasn't been given much love I think).

What we want is to be able to stabilise some interface that libraries like PJS can use to have transactions signed. If we have this, we can migrate things like PJS to using this new interface so that they aren't stuck relying on the PJS based one which has various issues including no standard way to handle new signed extensions.

How about an interface like this?:

/** The interface we'll expose */
type Interface = {
    /** Subscribe to acocunts. The callback fires whenever the list of acocunts changes */
    subscribeAccounts: (callback: (accounts: Account[]) => void) => void
    /** Sign a transaction, handing back the signed bytes ready to submit */
    signTransaction: (transaction: Transaction) => Promise<SignedTransaction>
}

/** Details about an available account */
type Account = {
    /** Often this will be a MultiAddress, or an AccountID20 on eth chains */
    address: Bytes
    /** Name to present with the account. */
    name: string
}

/** 
 * The information required to construct a transaction. We can ask to 
 * construct either a V4 or a V5 transaction.
 */
type Transaction = {
    output: 'v4-signed',
    address: Bytes,
    transactionExtensions: Bytes,
    callData: Bytes
} | {
    output: 'v5-signed',
    address: Bytes,
    transactionExtensionsVersion: number,
    transactionExtensions: Bytes,
    callData: Bytes
} | {
    output: 'v5-general',
    transactionExtensionsVersion: number,
    transactionExtensions: Bytes,
    callData: Bytes
}

/** Bytes representing the signed transaction, ready to submit */
type SignedTransaction = ArrayBuffer

/** Some binary data, in the form of a hex encoded string (0x prefixed) or raw bytes */
type Bytes = string | Uint8Array | ArrayBuffer;

The aim here is to be fairly minimal, but allow getting accounts (so that we know what the extension is able to sign) and signing transactions (which here allows you to ask for a V4 or V5 transaction and then provide the relevant details required to construct it.

• Is there anything that I'm missing?
• Can we simplify it even more?
• The account address here is whatever type the chain has configured, so that it can be put into the tx with the help of metadata. Is this reasonable? Should it be ss58 formatted or raw bytes as above (but that would then be an issue for eth chains)? Can extensions migrate to this easily enough?
• Should we also add a keypair: type KeypairType = 'ed25519' | 'sr25519' | 'ecdsa' | 'ethereum' to the account info, or is it enough to rely on the address type + metadata to decode and understand. I think we don't need the extra info.

\cc @josepot

[jsdw]

@ryanleecode asked about V5 general extensions, and this raises a good point.

To construct a V5 General extrinsic, the signature will live inside one of the extensions rather than as its own field. This means that a wallet would need to be able to insert the bytes for this extension into whatever extension bytes are given in order to sign it.

Given this, perhaps instead of providing the transaction extensions as one set of bytes, we should provide a key/value map from extension name to extension bytes like so:

wallet.signTransaction({
    output: 'v5-general',
    transactionExtensionsVersion: 0,
    transactionExtensions: {
        'CheckMortality': { encoded: '0xaabbcc', signerPayload: '0x1122' },
        'CheckNonce': '0x112233',
        // ...
    },
    callData: '0x1122334455aacc'
});

If a signed extension has "additional" data that makes it into the signer payload as well as data of its own, we can specify the data here with 'encoded' and 'signerPayload'. If one hex value is given we could default to assuming that there is no signerPayload, or we could deny this and just be explicit.

Then, a wallet can append these bytes together as dictated by the metadata (complaining if any extension bytes are required but not provided) and also add in the signature bytes as needed in the process in order to sign the thing.

What do you guys reckon?

[josepot]

I would like to propose that we first focus on defining the basic and most important interfaces before tackling the more complex ones. Specifically, I suggest we start by defining a new signature method to replace the problematic PJS signPayload.

In an ideal scenario, the extension would not only handle the creation of transactions but also manage the broadcasting process, while keeping the consumer informed about the transaction's status. This approach would allow us to have a single source of truth for the nonce, which is currently a significant challenge. However, this is beyond the scope of the interface I'm proposing here. For now, my focus is on making the current PJS extension interface library-agnostic.

Before proceeding, we need to agree on a couple of premises:

1. The extension can access the latest version of the chain's metadata for the transaction. We don't need to standardize how this is achieved at this point. For example, the PJS extension could implement an interface for registering a callback that the extension invokes whenever it needs to request the latest metadata from the dApp. Alternatively, adding light-client support to the PJS extension could allow it to obtain the latest metadata trustlessly.

2. The extension can access block information about the current tip of the transaction's chain. Again, it's not crucial to define how this is accomplished right now; what's important is that we agree that this premise can be fulfilled by all extensions. This is important because, to display the transaction's mortality properly to the user, the extension must have access to this information. This requirement is the main reason why the block number is passed as part of the payload in the problematic signPayload method.

Assuming we agree on these premises, I propose the following interface:

// Indicates that the string content must be a valid hexadecimal value
type HexString = string;

type BinaryData = Uint8Array | HexString;

type CreateTransaction = (
  from: BinaryData, // public key
  callData: BinaryData,
  signedExtensions: Record<string, BinaryData>
) => Promise<HexString>;

A few important notes:

1. The keys of the signedExtensions parameter must match the identifier property found in the extrinsic.signedExtensions entry of the metadata. The values are a SCALE encoded tuple containing (value, additionalSigned).

2. The extension should be able to identify the chain for which this transaction is for based on the values of the signedExtensions record (i.e thanks to the value of the CheckGenesis signed-extension, however this is something that could change in the future and it would be the responsibility of the extension to map that transaction to the correct chain based on the signedExtensions)

3. The consumer doesn't need to specify the type of transaction they want back; the extension can decide whether to assemble a version 5 or version 4 transaction. If the consumer wants to know the specifics of the transaction, they can decode it and examine it themselves.

4. The extension has the right to ask the user for additional input, allowing the user to modify some parameters (wrapping the tx in a multisig, changing the tip, changing the mortality, etc). When the dApp receives the final transaction, it can decide whether to broadcast it or not.

By starting with this fundamental interface, we can create a more flexible and library-agnostic foundation for transaction creation and signing, paving the way for future enhancements and more complex functionalities.

I want to re-iterate the fact that this suggestion is just for replacing the problematic PJS signPayload interface.

[jsdw]

Ok that makes sense. I had assumed that we'd also want access to account information in order that we know what "from" address to use (else we'll need to rely on the old interface or whatever for that stuff), but I also understand the reaosning behind just fixing this specific interface and not worrying about the rest yet so sounds good to me!