TransactionEffects V2

[lxfind]

Overview

TransactionEffects is one of the core data types in Sui. It contains information that summarizes the execution result of a transaction. The current version v1 is defined in https://github.com/MystenLabs/sui/blob/main/crates/sui-types/src/effects/effects_v1.rs. This type is evolved overtime and many fields are added to adapt new features introduced in the system. However these newly added fields lack of consistent design, which creates many problems. In this design we will propose TransactionEffectsV2, which has a coherent semantics around object changes, and will bring many benefits to the rest of the system.

Problems with TransactionEffectsV1

1. The semantics of effects are very confusing and hard to reason about.

Fields in effects data type are added individually without coherent design with each other. It becomes very difficult to reason about the semantics when we we look at them together. Very few people fully understand the precise semantics of each field in the effects related to objects. This is very problematic given how fundamental this data structure is. It also makes it very difficult to build any downstream component that depends on the semantics effects (e.g. state accumulator, object pruner and etc.). A few example questions to demonstrate the complexity of the fields:

• If an object is wrapped right after creation in the same transaction, will it show up in created or wrapped?
• If an object is deleted right after creation in the same transaction, will it show up in created or deleted?
• Will deleted / wrapped objects show up in modified_at_versions?
• Will unwrapped / unwrapped_then_deleted objects show up in modified_at_versions?

Overall, too much semantics are put in the effects that make it difficult to reason about and extend. We need to keep effects simple and only include primitive results around objects.

2. Dealing with effects is not efficient and often require unnecessary object store reads

When producing effects as well as consuming effects in various places, we often always have to read the object store again, which is quite inefficient and adds to I/O pressure:

• When we finish executing a transaction, in order to generate effects, we [consult](https://github.com/MystenLabs/sui/blob/main/sui-execution/latest/sui-adapter/src/programmable_transactions/context.rs#L743) the object store for each object that’s not in the input (which relies on whether an object exists as wrapped tombstone).
• In state accumulator, for each modified objects in each effects, we have to read the object store to get its old digest.

3. Storage inefficiency due to wrapped object tombstone

• We keep a wrapped tombstone whenever an object is wrapped.
• We can never garbage collect these wrapped tombstones
• Wrapped tombstones can leak: if object O1 is wrapped into O2, and we delete O2, O1 will remain as wrapped tombstone forever.
• Object pruner does not prune old wrapped tombstones: if we keep wrap and unwrap the same object, we can have many wrapped tombstones for the same object forever.

4. The effects data structure contains a lot of redundant data

• Redundant entries in multiple fields (same entry can show up in multiple fields).
• Redundant sequence number in all muated/created/wrapped entries (they are all lamport version)

TransactionEffects V2 design

There are a few guiding principles for the new effects design:

1. It should focus around how objects change in the store prior and after a transaction. This is what the core protocol ultimately cares about. Whether it is easy to do state accumulation based on the effects can be a good proxy measurement.
2. It should include only primitives, without embedding too much semantics in the effects. Any high-level semantics (e.g. wrapping) should not be a direct part of the effects, but could be derivable from the effects.
3. Each field contains precise and easy-to-understand meaning.
4. Minimum redundant information.

Organizing all object change information in a single vector could make it easy to see what IDs are touched/used in the transaction. It’s also most efficient in terms of effects size because there will never be duplication of object IDs.

Below is the new structure definition of TransactionEffects V2:

pub struct TransactionEffectsV2 {
    // These sets of fields are identical to TransactionEffectsV1
    status: ExecutionStatus,
    executed_epoch: EpochId,
    gas_used: GasCostSummary,
    /// The updated gas object reference, as an index into the `changed_objects` vector.
        /// Having a dedicated field for convenient access.
        gas_object_index: u16,
    transaction_digest: TransactionDigest,
    events_digest: Option<TransactionEventsDigest>,
    dependencies: Vec<TransactionDigest>,

    // These sets of fields are a redefinition of objects information.
        /// The version number of all the written Move objects by this transaction.
        lamport_version: SequenceNumber,
        /// Objects whose state are changed in the object store.
        changed_objects: Vec<(ObjectID, ObjectChange)>,
        /// Shared objects that are not mutated in this transaction. Unlike owned objects,
        /// read-only shared objects' version are not committed in the transaction,
        /// and in order for a node to catch up and execute it without consensus sequencing,
        /// the version needs to be committed in the effects.
        unchanged_shared_objects: Vec<(ObjectID, UnchangedSharedKind)>,
}

pub struct ObjectChange {
    // input_state and output_state are the core fields that's required by
    // the protocol as it tells how an object changes on-chain.
    /// State of the object in the store prior to this transaction.
    input_state: ObjectIn,
    /// State of the object in the store after this transaction.
    output_state: ObjectOut,

    /// Whether this object ID is created or deleted in this transaction.
        /// This information isn't required by the protocol but is useful for providing more detailed
        /// semantics on object changes.
        id_operation: IDOperation,
}

enum IDOperation {
        None,
       Created,
       Deleted,
}

pub type VersionDigest = (SequenceNumber, ObjectDigest);

/// If an object exists (at root-level) in the store prior to this transaction,
/// it should be Exist, otherwise it's NonExist, e.g. wrapped objects should be
/// NonExist.
pub enum ObjectIn {
    NotExist,
    Exist(VersionDigest),
}

pub enum ObjectOut {
    /// Same definition as in ObjectIn.
    NotExist,
    /// Any written object, including all of mutated, created, unwrapped today.
        ObjectWrite(ObjectDigest, Owner),
    /// Packages writes need to be tracked separately with version because
    /// we don't use lamport version for package publish and upgrades.
        PackageWrite(VersionDigest),
}

enum UnchangedSharedKind {
        /// Read-only shared objects from the input. We don't really need ObjectDigest
        /// for protocol correctness, but it will make it easier to verify untrusted read.
        ReadOnlyRoot(VersionDigest),
        /// Child objects of read-only shared objects. We don't need this for protocol correctness,
        /// but having it would make debugging a lot easier.
        ReadOnlyChild(VersionDigest),
        /// Already deleted shared objects.
        Deleted(SequenceNumber),
}

To map the semantics to today’s terms:

ObjectIn	ObjectOut	id_created	id_deleted	Description
NotExist	NotExist	Y	N	Create-then-wrapped
NotExist	NotExist	N	Y	Unwrap-then-deleted
NotExist	ObjectWrite	Y	N	Created
NotExist	ObjectWrite	N	N	Unwrapped
NotExist	PackageWrite	Y	N	Package publish
Exist	NotExist	N	Y	Deleted
Exist	NotExist	N	N	Wrapped
Exist	ObjectWrite	N	N	Mutated
Exist	PackageWrite	N	N	Package upgrade

As for state accumulation, the algorithm becomes rather simple: for each object in changed_objects, if its ObjectIn is Exist, remove it from the accumulator; if its ObjectOut is not NotExist, add it to the accumulator.

https://github.com/MystenLabs/sui/pull/12778 provides an initial implementation of the type definition.

RPC/SDK changes

To make the change smooth, we will keep the RPC effects type unchanged initially, while introducing a new RPC type to capture the shape of effects V2. When queried with effects V2 type, the server will translate effects V2 to the existing RPC effects type. There will be some small transaction losses, which will be described in more detail latter.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.