Build test suite

[AdamISZ]

Some initial thoughts:

• Use pytest and re-use build/teardown code from JM

• Further to the above, at a minimum, realistic tests require at minimum two separate regtest instances so that the wallets are correctly isolated. Have already done this manually (started with this, have made my own example configs).

• Alternative to the above: build dummy blockchain instances. In principle this is simple, in practice it's hard to get right since BCI must return utxos and wallet updates at various steps (and support scanning for secrets). This may not be workable (and is of course less realistic, too).

• The main focus of the tests is each backout scenario. They must be tested both in-run and after crash/shutdown, using persisted json state, for both sides.

[AdamISZ]

Note that since #2 is a pretty big change, it would make no sense to start this until that is finished.

[AdamISZ]

A first step in 400d0454f110dbf77c64c59b1c0e3743c4c288c3

This runs both sides fairly simply (in one reactor), tested working after the following commit, for both cooperative and at least some backout modes.

Next steps:

1. Make a clean set of checks/asserts for whether run is successful or not
2. Start building further test cases for each backout case in-run.
3. More difficult: build test cases for backouts on shutdown and restart.
4. Also difficult: make multiple runs work in pytest considering nasty behaviour w.r.t. twisted reactor - more than one reactor in the same run is not supported iirc.

[AdamISZ]

b40d9b1d152cb533e0d83471ed5c85bdfebab0bb now has a fairly simple method to test all cases where Alice is malicious or fails erroneously, in run.

W.r.t. 4, found that using a config var --runtype=foo on the command line with pytest is a workable model: the test code now picks up a specific modified Alice* class derived from CoinSwapAlice that injects some kind of error or failure into one callback during state machine execution. The customised Alice classes are in test/bad_participant.py. This same model should work fine for Carol side also. All that leaves is to have an automation of calling the test with each possible value of runtype, this can be done in a shell script and return values can be checked to see if tests passed or not.

So now it seems: (1) is done, (2) need to complete for Carol side being malicious, (4) I will probably ditch and just use the above mechanism with a shell script, (3) is going to be hard, will defer to another issue, once this is done, but at least make sure manual testing can be done.

[AdamISZ]

6fc8191fc588f2c201ef9296d55219a066cfe3b1 now Carol as well as Alice side maliciousness is covered, it also provides a bash script run_all.sh addressing (4) (as above). I now think (3) can be done fairly easily, so will do that before closing this.

[AdamISZ]

The recovery from crash tests are now done in 3989df1fe6b3f48dac167147d72e8da3f8ddb17c

These use a BadStateMachine class to inject a failure, stopping the state machine, at a configured state. Then the failure_injection method of the modified BadAlice, BadCarol classes is called, running load() and backout(), as is the case when the --recover mode is chosen. This is not a watertight test of real world conditions, but pretty good.

This adds another ~ 20 test cases to the existing ~20 cases for in-run backout. The two sets of 20 are now runnable with run_all.sh and run_all_recover.sh, with the blockchain parameters configurable as command line arguments.

Closing now as this is a reasonable baseline test suite. See comments above about malleability/double spend cases, which I'll defer to a new issue.