iotaledger / iota

Apache License 2.0
16 stars 7 forks source link

[transactional-test-runner] Document syntax of tasks in transactional tests #1979

Open kodemartin opened 3 months ago

kodemartin commented 3 months ago

Description

Transactional tests simulate network operations through the framework exposed in iota-transactional-test-runner. The framework is actually built on top of the more generic move-transactional-test-runner.

This currently used in the following tests:

$ cargo tree -i iota-transactional-test-runner
iota-transactional-test-runner v0.1.0 (/home/kodemartin/projects/kinesis/crates/iota-transactional-test-runner)
[dev-dependencies]
├── iota-adapter-transactional-tests v0.1.0 (/home/kodemartin/projects/kinesis/crates/iota-adapter-transactional-tests)
├── iota-graphql-e2e-tests v0.1.0 (/home/kodemartin/projects/kinesis/crates/iota-graphql-e2e-tests)
└── iota-verifier-transactional-tests v0.1.0 (/home/kodemartin/projects/kinesis/crates/iota-verifier-transactional-tests)

Syntax

The framework introduces an ad-hoc syntax for defining network related operations/tasks as an extension to move/mvir files. The syntax uses comments with the //# prefix that bring about additional syntax rules for parsing additional arguments for the tasks. For example,

//# init --protocol-version 1 --simulator --accounts C

//# create-checkpoint

//# advance-epoch

//# programmable --sender C --inputs 10000000000 @C
//> SplitCoins(Gas, [Input(0)]);
//> TransferObjects([Result(0)], Input(1))

//# run 0x3::iota_system::request_add_stake --args object(0x5) object(3,0) @validator_0 --sender C

//# create-checkpoint

//# advance-epoch

// run another transaction
//# programmable --sender C --inputs 10000000000 @C
//> SplitCoins(Gas, [Input(0)]);
//> TransferObjects([Result(0)], Input(1))

//# create-checkpoint

//# advance-epoch

//# run-graphql
{
  epoch(id: 2) {
    validatorSet {
      totalStake
      activeValidators {
        nodes {
          name
        }
      }
      validatorCandidatesSize
      inactivePoolsId
    }
    totalGasFees
    totalStakeRewards
    fundSize
    fundInflow
    fundOutflow
    netInflow
    transactionBlocks {
      nodes {
        kind {
          __typename
        }
        digest
      }
    }
  }
}

Motivation

It seems that this syntax is not documented, and hence if one were to actually read or write new transactional tests, one should reverse engineer the syntax rules.

It will be thus helpful to identify and document the set of tasks that can be defined in transactional tests.

Get started

Parsing

Parsing takes place through the taskify function that accepts a generic Command type.

https://github.com/iotaledger/iota/blob/e6f96d540956ca7c445051cd6cd455d37a8f0a92/external-crates/move/crates/move-transactional-test-runner/src/tasks.rs#L33

Generic commands

The generic Command type used is the following:

https://github.com/iotaledger/iota/blob/e6f96d540956ca7c445051cd6cd455d37a8f0a92/external-crates/move/crates/move-transactional-test-runner/src/tasks.rs#L261-L273

This defines a set of main tasks:

and a general class of sub-commands subcommand that can be extended in implementors.

Concrete implementation

print-bytecode uses the following type

https://github.com/iotaledger/iota/blob/e6f96d540956ca7c445051cd6cd455d37a8f0a92/external-crates/move/crates/move-transactional-test-runner/src/tasks.rs#L206-L212

For the rest of the tasks the parsing details are defined in IotaTestAdapter which implements MoveTestAdapter trait:

https://github.com/iotaledger/iota/blob/e6f96d540956ca7c445051cd6cd455d37a8f0a92/crates/iota-transactional-test-runner/src/test_adapter.rs#L173-L178

The set of subcommands is defined through

https://github.com/iotaledger/iota/blob/e6f96d540956ca7c445051cd6cd455d37a8f0a92/crates/iota-transactional-test-runner/src/args.rs#L207-L223

### Tasks
- [ ] https://github.com/iotaledger/iota/issues/1997
- [ ] Document `Init` command.
- [ ] Document `Run` command.
- [ ] Document `Publish` command.
- [ ] Document `PrintBytecode` command.
- [ ] Document `ViewObject` subcommand.
- [ ] Document `TransferObject` subcommand.
- [ ] Document `ConsensusCommitPrologue` subcommand.
- [ ] Document `ProgrammableTransaction` subcommand.
- [ ] Document `UpgradePackage` subcommand.
- [ ] Document `StagePackage` subcommand.
- [ ] Document `SetAddress` subcommand.
- [ ] Document `CreateCheckpoint` subcommand.
- [ ] Document `AdvanceEpoch` subcommand.
- [ ] Document `AdvanceClock` subcommand.
- [ ] Document `SetRandomState` subcommand.
- [ ] Document `ViewCheckpoint` subcommand.
- [ ] https://github.com/iotaledger/iota/issues/4201
- [ ] Document `ForceObjectSnapshotCatchup` subcommand.
- [ ] Document `Bench` subcommand.

Requirements

kodemartin commented 3 months ago

A hack to get the cli help for each supported task is try to run single task with <task> --help. The test will fail with the usage output.

For example, create a file init.move inside iota-graphql-e2e-tests/tests/ with

//# init --help

Running the test will fail, but will also produce

Usage: task init [OPTIONS]

Options:
      --accounts <ACCOUNTS>...

      --protocol-version <PROTOCOL_VERSION>

      --max-gas <MAX_GAS>

      --shared-object-deletion <SHARED_OBJECT_DELETION>
          [possible values: true, false]
      --simulator

      --custom-validator-account

      --reference-gas-price <REFERENCE_GAS_PRICE>

      --default-gas-price <DEFAULT_GAS_PRICE>

      --object-snapshot-min-checkpoint-lag <OBJECT_SNAPSHOT_MIN_CHECKPOINT_LAG>

      --object-snapshot-max-checkpoint-lag <OBJECT_SNAPSHOT_MAX_CHECKPOINT_LAG>

      --flavor <FLAVOR>

      --addresses <NAMED_ADDRESSES>...