search

IntersectMBO / cardano-node-tests

System and end-to-end (E2E) tests for cardano-node.
https://tests.cardano.intersectmbo.org/
Apache License 2.0
53 stars 28 forks source link

readme

README for cardano-node-tests

Ruff Code style: black

System and end-to-end (E2E) tests for cardano-node.

Check this documentation for more details.

Running tests using Github Actions

The easiest way to run the tests is by using Github Actions.

  1. fork this repository
  2. enable Github Actions in your fork ("Settings" / "Actions" / "General" / "Actions permissions", check "Allow all actions and reusable workflows")
  3. go to "Actions", select "01 Regression tests" (or "02 Regression tests with db-sync")
  4. select "Run workflow"

Run workflow

Running tests using Nix

  1. install and configure nix, follow cardano-node documentation

  2. clone this repo

  3. run the tests

    ./.github/regression.sh

NOTE

When using CI_BYRON_CLUSTER, it takes ~ 30 minutes for local cluster instance to get from Byron to Babbage. If it seems that tests are stuck, they are likely just waiting for local cluster instances to be fully started.

Running individual tests on persistent local cluster using Nix

Sometimes it is useful to run individual tests and keep the local cluster running in between test runs.

  1. run nix shell that has all the needed dependencies

    nix flake update --accept-flake-config --override-input cardano-node "github:IntersectMBO/cardano-node/master"  # change `master` to rev you want
nix develop --accept-flake-config .#venv

  2. prepare testing environment

    source ./prepare_test_env.sh babbage  # 'conway' is also supported

  3. start the cluster instance

    ./dev_workdir/babbage_fast/start-cluster

  4. run some test

    pytest -s -k test_minting_one_token cardano_node_tests/tests/tests_plutus
# or run some tests and see all the executed `cardano-cli` commands
pytest -s --log-level=debug -k test_minting_one_token cardano_node_tests/tests/tests_plutus

  5. stop the cluster instance

    ./dev_workdir/babbage_fast/stop-cluster

Variables for configuring testrun

Tests execution can be configured using env variables.

When running tests using the ./.github/regression.sh script, you can also use

For example:

Local usage for tests development (useful only for tests developers)

Install and configure nix, follow cardano-node documentation. Install and configure poetry, follow Poetry documentation.

Preparing Python virtual environment

Create a Python virtual environment (requires Python v3.8 or newer) and install this package together with development requirements:

./setup_dev_venv.sh

Running development cluster

When running tests, the testing framework starts and stops cluster instances as needed. That is not ideal for test development, as starting a cluster instance can take up to several epochs (to get from Byron to Babbage). To keep the Cardano cluster running in between test runs, one needs to start it in 'development mode':

  1. cd to 'cardano-node' repo

    cd ../cardano-node

  2. update and checkout the desired commit/tag

    git checkout master
git pull origin master
git fetch --all --tags
git checkout tags/<tag>

  3. launch devops shell

    nix develop .#devops

  4. run fresh login shell on top of the current nix shell (to get the correct environment variables)

    /bin/bash --login

  5. cd back to 'cardano-node-tests' repo

    cd ../cardano-node-tests

  6. activate virtual env

    poetry shell

  7. add virtual env to PYTHONPATH

    export PYTHONPATH="$(echo $VIRTUAL_ENV/lib/python3*/site-packages)":$PYTHONPATH

  8. set env variables

    export CARDANO_NODE_SOCKET_PATH="$PWD/dev_workdir/state-cluster0/bft1.socket" DEV_CLUSTER_RUNNING=1
mkdir -p "${CARDANO_NODE_SOCKET_PATH%/*}"

  9. prepare cluster scripts for starting local cluster directly in Babbage era

    prepare-cluster-scripts -c -d dev_workdir/babbage_fast -s cardano_node_tests/cluster_scripts/babbage_fast/

  10. start the cluster instance in development mode

    ./dev_workdir/babbage_fast/start-cluster

After the cluster starts, keys and configuration files are available in the ./dev_workdir/state-cluster0 directory. The pool-related files and keys are located in the nodes subdirectory, genesis keys in the shelley and byron subdirectories, and payment address with initial funds and related keys in the byron subdirectory. The local faucet address and related key files are stored in the addrs_data subdirectory.

Restarting development cluster

To restart the running cluster (eg, after upgrading cardano-node and cardano-cli binaries), run:

./scripts/restart_dev_cluster.sh

NOTE

Restaring the running development cluster is useful mainly when using the "babbage" start scripts (not the "babbage_fast" version). It takes ~ 30 minutes for the local cluster instance to get from Byron to Babbage. Starting local cluster using the "babbage_fast" version takes less than 1 minute.

Checking the development environment

To check that the development environment was correctly setup, run the ./check_dev_env.sh script.

$ ./check_dev_env.sh
'cardano-node' available: ✔
'cardano-cli' available: ✔
'python' available: ✔
'pytest' available: ✔
'nix-shell' available: ✔
'jq' available: ✔
'supervisord' available: ✔
'supervisorctl' available: ✔
'bech32' available: ✔
inside nix shell: ✔
in repo root: ✔
DEV cluster: ✔
python works: ✔
in python venv: ✔
venv in PYTHONPATH: ✔
cardano-node-tests installed: ✔
pytest works: ✔
same version of node and cli: ✔
socket path set: ✔
socket path correct: ✔
socket path exists: ✔
cluster era: babbage
transaction era: babbage
using dbsync (optional): ✔
dbsync available: ✔
P2P network (optional): -

Running individual tests

Example:

pytest -k "test_name1 or test_name2" cardano_node_tests
pytest -m "not long" cardano_node_tests
pytest -m smoke cardano_node_tests/tests/test_governance.py

Running linters

It is sufficient to activate the Python virtual environment before running linters, development cluster is not needed:

  1. activate virtual env

    poetry shell

  2. run linters

    make lint

Installing cardano-clusterlib in development mode

Sometimes it is useful to test local changes made to cardano-clusterlib. To install cardano-clusterlib in development mode:

  1. activate virtual env

    poetry shell

  2. enable legacy behavior (needed by mypy and pylint linters)

    export SETUPTOOLS_ENABLE_FEATURES="legacy-editable"

  3. update virtual env (answer 'y' to question "Install into the current virtual env? [y/N]")

    ./setup_dev_venv.sh

  4. uninstall cardano-clusterlib installed by poetry

    pip uninstall cardano-clusterlib

  5. cd to 'cardano-clusterlib-py' repo

    cd ../cardano-clusterlib-py

  6. install cardano-clusterlib in development mode

    pip install -e .

  7. cd back to 'cardano-node-tests' repo

    cd -

  8. check that you are really using cardano-clusterlib files from your local repo

    python -c 'from cardano_clusterlib import clusterlib_klass; print(clusterlib_klass.__file__)'

Note that after you run poetry install (eg through running ./setup_dev_venv.sh), poetry will reinstall cardano-clusterlib. If you want to keep using cardano-clusterlib in development mode, you'll need to repeat the steps above.

Updating dependencies using poetry

Edit pyproject.toml and run

./poetry_update_deps.sh

Building documentation

Build and deploy documentation:

./deploy_doc.sh

Contributing

Install this package and its dependencies as described above.

Run pre-commit install to set up the Git hook scripts that will check your changes before every commit. Alternatively, run make lint manually before pushing your changes.

Follow the Google Python Style Guide, with the exception that formatting is handled automatically by Black (through the pre-commit command).

See the CONTRIBUTING document for more details.