Open plaidfinch opened 3 years ago
@jakinyele what is the current status of this? Are we moving this to milestone 4?
Correct, we won't have any automated scripts to support end-to-end tests until milestone 4. Will create a new issue for testing goal specifically for milestone 3.
The goal is to develop a script for end-to-end testing zkChannels lifecycle (ie establish, pay, and close) on
Update 1/10/22: The current version of the test script executes many code paths, but it does not effectively compare them to the expected outcomes. In particular, it does not distinguish "happy paths" from test cases that should fail, and it does not react to non-abortive failures in the chain watchers (#338).
I propose an updating testing framework in Rust that uses the standard test conventions (e.g. lives in
test/
) and solves these issues. Like the existing script, it wouldEach test is described as a struct with a list of pairs of an operation (e.g. establish, pay, pay all, expiry) and an expected outcome (customer status, merchant status, whether an error should be raised).
The tests will run the setup phase once, then run each test. Every test needs to monitor the output from the three processes (server, watcher, and operation) and identify errors relevant to itself; this may become easier with #91. It will also check the expected outcome against the current status (e.g. by calling
list
for each party).The tests cannot be run in parallel with the current Tezos instantiation per #282, #212. We can prevent this with a testing flag. If we do eventually parallelize testing, we will need to define a way for the test to identify errors that belong to itself, as the server and watcher may be generating errors for multiple tests in parallel.
Ideally, this will confer the following benefits: