spacemeshos / SMIPS

Spacemesh Improvement Proposals
https://spacemesh.io
Creative Commons Zero v1.0 Universal
7 stars 1 forks source link

Transaction Abstraction Model and Terminology #77

Open lrettig opened 2 years ago

lrettig commented 2 years ago

Overview

Transactions play a central role in the Spacemesh network and protocol. They are passed up and down throughout several layers of abstraction in our protocol design and implementation. This proposal is an attempt to unify the terminology and design around this abstraction model.

Goals and motivation

Specify a single, universal, concrete abstraction model and a set of terminology for transactions to be used throughout the protocol design and implementation. This should make it easier for code and documentation to refer concretely to one or more layers in the stack using standard, accepted terminology.

High-level design

Similar to the OSI model for the application stack, transactions in Spacemesh can be interpreted according to a multi-layer conceptual model. The model partitions the flow of transactions (and the state transitions they contain) into several abstraction layers, from the wire format (conceptually at the bottom of the stack) to the application layer (at the top).

Prior art

We are unaware of any similar model in the blockchain/cryptocurrency/Web3 world.

Spacemesh

Proposed implementation

Layer 0: Wire layer

At this layer, a transaction is encoded as a p2p message that's sent directly or gossiped to the Spacemesh p2p network. It contains the information necessary to route it to its destination (in the case of direct send), and a signature to verify the sender and ensure it hasn't been tampered with:

WireTransaction := OriginSignature || RoutingData? || TransactionEnvelope

Handled by: all p2p network participants, at the p2p layer only

Layer 1: Envelope layer

At this layer, a transaction is interpreted by the node as an opaque transaction of a certain type, without regard to any internal data (such as origin, principal, calldata, signature, etc.). It consists of a type indicator followed by an opaque byte array whose interpretation depends on the type:

TransactionEnvelope := TransactionType || TransactionPayload

We define the following transaction types:

Handled by: miners, all clients reading block data

Layer 2: Payload layer

This is the first layer which involves interpretation inside the VM. Interpretation at this layer depends on the type indicator in the TransactionEnvelope. At this layer, a transaction is interpreted by the VM as having a VM type.

TransactionPayload := VMType || PrincipalAddress || VMPayload

Note that the TransactionPayload does not contain a signature because different principal accounts may choose to interpret signatures in different ways (and, theoretically, may not even require one). The same is true for gas parameters.

Handled by: SVM top-level transaction handler

Layer 3: Dispatch layer

At this layer, the contents of the VMPayload are interpreted based on the VMType. The VM is responsible for directly handling one special-case type, self-spawn. For all other transaction types, the VMPayload is passed directly into the PrincipalAddress execution handler method as-is.

Self-spawn

The VM is responsible for partially unpacking and handling self-spawn transactions.

If VMType == SelfSpawn:
  DestinationAddress := hash(VMPayload)
  VMPayload := TemplateAddress || ExecutionCalldata

In this case, the VM calculates the DestinationAddress (based on VMPayload), loads the template referenced by TemplateAddress, tentatively deploys it to the DestinationAddress, then calls its constructor and passes in the ExecutionCalldata.

Base case

Nothing happens at this layer. VMPayload is passed as-is into the PrincipalAddress execution handler method.

Else:
  VMPayload := ExecutionCalldata

Handled by: SVM core dispatcher

Layer 4: Execution layer

At this layer, the ExecutionCalldata is unpacked and verified/executed by the principal account handler. Principal accounts may be implemented in many ways, as long as they conform to the principal account API (see #78). As such, they may choose to interpret ExecutionCalldata in many ways. However, standard Spacemesh principal accounts expect the following types of transactions, and in most cases, they should expect ExecutionCalldata to include standard fields such as Nonce, MaxGas, GasPrice, etc.

deploy: used to deploy a new smart contract template.

ExecutionCalldata[deploy] := Signature || Nonce || MaxGas || GasPrice || Code
DestinationAddress := hash(Code)

The handler should verify the transaction on the basis of the Signature, calculate a new DestinationAddress based on the Code, and deploy the code as a new template at this address.

spawn: used to spawn a new smart contract instance, based on a deployed template.

ExecutionCalldata[spawn] := Signature || Nonce || MaxGas || GasPrice || TemplateAddress || CallData
DestinationAddress := hash(TemplateAddress || CallData)

The handler should verify the transaction on the basis of the Signature, calculate a new DestinationAddress based on TemplateAddress and CallData, deploy the template to the address, then call the constructor of the newly-spawned contract with the CallData.

call: used to call into a spawned smart contract instance.

ExecutionCalldata[call] := Signature || Nonce || MaxGas || GasPrice || CallData

The handler should verify the transaction on the basis of the Signature, then call the execute handler on the Principal, passing in CallData.

Handled by: Principal account handlers, including verify() and execute().

Layer 5: Application layer (optional)

At this layer, the principal account handler may, on the basis of the received CallData, call into a selected method, either locally or on an external target contract. On this basis any smart contract, including user-deployed applications, finally receives an inbound call from a principal account handler.

This layer is marked "optional" because not all transactions will be carried as high as this layer, their execution having ceased at the previous layer.

CallData := TargetAddress? || TargetSelector || TargetCallData?

The handler should look up the smart contract referred to by TargetAddress (if specified; if not, it may be assumed that the target is the principal contract itself), call the method selected by TargetSelector, and pass in TargetCallData (if any).

As a concrete example, the case of a "simple coin send" would consist of CallData that looks like the following:

CallData := TargetSelector="send" || TargetCallData
TargetCallData := RecipientAddress || Amount

Handled by: principal account (non-handler) methods, applications deployed as smart contracts

Implementation plan

N/A, this proposal specifies a design only

Questions

Dependencies and interactions

All existing and future Spacemesh documentation should reflect this model, and where possible it should also be reflected in implementations.

Stakeholders and reviewers

Testing and performance

N/A

noamnelke commented 2 years ago

In general I think it would be useful to show the structure of the entire transaction to make it easier to read the spec. It can be as simple as the following (which I did so I can see the whole thing at once):

WireTransaction := OriginSignature || RoutingData? || TransactionType || VMType || PrincipalAddress || VMPayload

Although it may be worth constructing some visual representation including the layers and the fields in each one.

+-------------------------+
| Layer 0: Wire           |
|   OriginSignature       |
|   RoutingData           |
+-------------------------+
| Layer 1: Envelope layer |
|   TransactionType       |
+-------------------------+
...

Layer 0: Wire layer

I don't remember discussing RoutingData or "direct send" of transactions... I can't see why we would need either of these and you don't explain the motivation.

I don't think there should be a signature at this layer. The transaction is validated as a whole by the Verify method of the principal's template which should include signature[s] verification.

Layer 1: Envelope layer

👍🏻

Layer 2: Payload layer

What is the VMType? Do you mean Deploy/Spawn/Call?

I'm actually thinking that we can do better than to "waste" an entire byte on this. We put a MethodSelector byte after the PrincipleAddress and it will have 2 reserved values for Deploy (0x00) and Spawn (0x01).

TransactionPayload := PrincipalAddress || MethodSelector || VMPayload

Deploy expects VMPayload to be TemplateCode || ValidationData.

Spawn expects VMPayload to be Template || ImmutableState || ValidationData. The spawned account address will be hash(VMPayload). If the principle is a stub (unspawned account) then we first validate that MethodSelector is 0x01 and that hash(VMPayload) is the principle address - otherwise the transaction is invalid. Spawned accounts can spawn other accounts as well by just sending a VMPayload for an account that is either a stub or doesn't exist.


This design implies that we only support a very specific constructor - the one used for self-spawn, which only gets the immutable storage as input and stores it as-is. It also doesn't support atomic "spawn & fund". This simplifies the design and I think that any use-case that requires running code at spawn time or funding atomically can be achieved with a factory (an account that spawns and initializes other accounts). I think this trade-off is worth it, but I'm open to discuss it.


Our genesis reference contracts will implement a method called ExternalCall (0x02) that expects VMPayload to be Target || Method || Amount || Arguments || ValidationData. Method must be >1, Amount to send along with the call, Arguments are passed into the target method.

Values >2 will be mapped to other template-specific methods, like a simple Send.

Layer 3: Dispatch layer

👍🏻

Layer 4: Execution layer

I realize this is just an example, since every template can define its own structure, but I think the reference implementation should be as follows:

ExecutionCalldata[call] := CallData || ValidationData
ValidationData := Nonce || MaxGas || GasPrice || WitnessData
WitnessData := Signature

I would wrap ValidationData and explain in the text that this part of the ExecutionCalldata is defined by the principle template.

In our reference templates,WitnessData should be at the very end of the entire transaction, so that we can hash a contiguous buffer of bytes for signature validation. In case of a single sig, as we have today, it will be all bytes of the Wire Transaction up to the last 64 - the length of the signature - but it could also be more signatures or something else entirely, as defined by the template.

Putting the CallData in the beginning is mostly important to be consistent with what I have in mind for Spawn transactions, as explained above:

ExecutionCalldata[spawn] := TemplateAddress || CallData || ValidationData
DestinationAddress := hash(TemplateAddress || CallData)

This means we don't have to include the TemplateAddress twice.


Here's a summary of what I think we should do, including some naming suggestions:

Transaction := TransactionType || TransactionBody || ValidationData
TransactionType := 0x00 (reserved for future use)
TransactionBody := PrincipalAddress || MethodSelector || MethodArguments
MethodSelector := 0x00 (Deploy) | 0x01 (Spawn) | MethodID
MethodID := >0x01

if MethodSelector == 0x00 (Deploy):
  MethodArguments := TemplateCode
if MethodSelector == 0x01 (Spawn):
  MethodArguments := TemplateAddress || ImmutableState

The following is defined by the template and is just a simple example:

ValidationData := Nonce || MaxGas || GasPrice || WitnessData
WitnessData := Signature
lrettig commented 2 years ago

Layer 0: Wire layer

I don't remember discussing RoutingData or "direct send" of transactions... I can't see why we would need either of these and you don't explain the motivation.

I don't think there should be a signature at this layer. The transaction is validated as a whole by the Verify method of the principal's template which should include signature[s] verification.

This is all handled in the p2p code in the node. It's orthogonal to transaction handling. I'm just including it here for the sake of completeness. It is the "bottom layer", from the perspective of the Spacemesh node and protocol.

Layer 2: Payload layer

What is the VMType? Do you mean Deploy/Spawn/Call?

It's really just SelfSpawn vs. other. All other types are handled dynamically, inside the principal contract handlers (layer 4).

We put a MethodSelector byte after the PrincipleAddress and it will have 2 reserved values for Deploy (0x00) and Spawn (0x01).

But Deploy can also be handled in an abstract fashion, inside the principal contract handlers (layer 4). The only special case that needs to be handled by the VM before calling into the principal contract is the self-funded spawn.

In our reference templates,WitnessData should be at the very end of the entire transaction, so that we can hash a contiguous buffer of bytes for signature validation. In case of a single sig, as we have today, it will be all bytes of the Wire Transaction up to the last 64

I'm a little stuck on how signing should work. The issue is that the signature lives in the innermost data structure, as you rightly point out, but it should be a signature across all of the tx bytes, which breaks multiple abstraction boundaries - it means the innermost layer needs access to the entire tx byte array (even though it, e.g., doesn't know how to interpret the portions corresponding to higher layers). Need to give this some more thought/we need to discuss this further.

noamnelke commented 2 years ago
Transaction := TransactionType || TransactionBody
TransactionType := 0x00 (reserved for future use)
TransactionBody := PrincipalAddress || TransactionVerb || Arguments || ValidationData
TransactionVerb := 0x00 (Deploy) | 0x01 (Spawn) | 0x02 (Call)

if TransactionVerb == 0x00 (Deploy):
  Arguments := GasMeteringMode || TemplateCode
if TransactionVerb == 0x01 (Spawn):
  Arguments := TemplateAddress || ImmutableState
if TransactionVerb == 0x02 (Call):
  Arguments := MethodSelector || MethodArguments

The following is defined by the template and is just a simple example:

ValidationData := Nonce || MaxGas || GasPrice || WitnessData
WitnessData := Signature