cue-unity / unity

unity - run experiments/regression tests on CUE modules
Apache License 2.0
41 stars 5 forks source link

cmd/unity: should we run tests in a copy of the repository that contains a CUE module? #45

Open myitcv opened 3 years ago

myitcv commented 3 years ago

Currently, unity runs tests declared in the cue.mod/tests manifest against a copy of the repository that contains the CUE module under test (setting an appropriate working directory at the start of the test).

This allows the tests declared in the manifest to run concurrently, but most importantly for us to be sure the tests are independent of each other.

However, it does mean that any side effects of a test (e.g. the results of cue export) are not reflected within the repository from which the tests originated.

This might well prove to be more of a pain than the pain of tests interfering with each other.

This raises the question of the overlap between:

It's not clear at this stage where the lines need to be drawn (if at all).