This proposal suggests adding an in-repo automated suite of e2e tests that can be expanded upon and help reduce the effort needed to increase the confidence of a specific release.
Reduce the time necessary to run tests in the release cycle
Non-goals
Increase coverage from that of existing tests
Fix any defects revealed by the tests
Mimic the tests' exact behavior if it would require changing pocket-core code to enable interfacing properly
Components:
Network Launcher
Pocket Client
Pocket Server
Network Configurations
Tests
cucumber style feature files, with scenarios written in gherkin
step implementation for scenario creation and execution
new external dependency: godog, to parse feature files and run scenarios
Details
The code to be introduced would depend on code from pocket-core, but would not affect any builds.
The complexity introduced would be self-contained in its packages, which would live under the e2e folder at the top level.
The Network Launcher compiles pocket-core, establishes datadirs for all the defined nodes, and starts the network. It contains in runtime the information defined in the network configuration.
Network configurations allow defining a genesis file, node configurations, and hold the private keys for the actors of the network.
The Pocket Client and Pocket Server can be run with any command-line arguments. We get to read their full stdout and stderr, which allows for inspecting state transitions as evidenced by, for example, the debug log.
The tests are defined in a twofold process:
Design and implementation of the scenarios, choosing or defining the "steps" required to verify that the system is behaving the way we want. This should be biased towards reusing existing steps. This is done in gherkin.
Design and implementation of steps so they work together properly. This includes designing the state that should be kept from one step to the other and how it will be made available to a future step. This is done in go.
This proposal suggests adding an in-repo automated suite of e2e tests that can be expanded upon and help reduce the effort needed to increase the confidence of a specific release.
Goals
Non-goals
Components:
Details
The code to be introduced would depend on code from pocket-core, but would not affect any builds. The complexity introduced would be self-contained in its packages, which would live under the e2e folder at the top level.
The Network Launcher compiles pocket-core, establishes datadirs for all the defined nodes, and starts the network. It contains in runtime the information defined in the network configuration.
Network configurations allow defining a genesis file, node configurations, and hold the private keys for the actors of the network.
The Pocket Client and Pocket Server can be run with any command-line arguments. We get to read their full stdout and stderr, which allows for inspecting state transitions as evidenced by, for example, the debug log.
The tests are defined in a twofold process: