The integration testing framework has grown to be quite complex, and although originally used to run automated tests in CI, it has become useful to run scenarios manually and interactively sometimes on deployed test chains/playgrounds to apply certain state changes.
The framework is not documented and very hard to use in the latter case especially when multiple scenarios are executed and when it comes to re-using the generated keys from a previous run.
Additionally many of the scenarios, flows and fixtures might still have implied assumptions about the state of the chain when executed which can cause them to fail when used "at the wrong time". The scenarios are not configurable, in general no command line arguments are used nor is user interaction expected.
Some docker services such as storage infra that depend on the council, lead and worker being setup with these scripts are expecting a pre-defined worker id, and keys to be set. This should be improved so that any relevant configuration options for the services can be taken from the output of the scenarion run.
The integration testing framework has grown to be quite complex, and although originally used to run automated tests in CI, it has become useful to run scenarios manually and interactively sometimes on deployed test chains/playgrounds to apply certain state changes.
The framework is not documented and very hard to use in the latter case especially when multiple scenarios are executed and when it comes to re-using the generated keys from a previous run.
Additionally many of the scenarios, flows and fixtures might still have implied assumptions about the state of the chain when executed which can cause them to fail when used "at the wrong time". The scenarios are not configurable, in general no command line arguments are used nor is user interaction expected.
Some docker services such as storage infra that depend on the council, lead and worker being setup with these scripts are expecting a pre-defined worker id, and keys to be set. This should be improved so that any relevant configuration options for the services can be taken from the output of the scenarion run.