Open jvgaultney opened 2 years ago
In our discussion there was some disagreement about whether the command line should completely override whatever is in the wscript (which would be simpler), but I think it would be better for it not to override. One option for making that simpler would be have two additional tests folder lists: EXTRATESTS defined in the wscript and MORETESTS as a command0line parameter. That would take care of the scenario where EXTRATESTS is something that should always be run and MORETESTS points to a folder of temporary ones only needed at the moment.
I'm not sure we need to introduce a new wscript variable.
In the same way the wscript DOCDIR
was expanded to handle a list of documentation directories, e.g.:
DOCDIR=['documentation', 'web']
Why not expand functionality of TESTDIR
to take multiple test directories (some of which could be outside the project)? e.g.:
TESTDIR=['tests', '../additional/test/folder', '../another/one']
Good idea! It's also good to be consistent about how wscript handles paths.
But if you want an override-able default, then we would also need EXTRATESTDIR. So, I propose:
I can confirm TESTDIR and EXTRATESTDIR can now hold multiple path lists and test data in these multiple folders are taken into account by the smith tests targets.
Please add support for running tests using test files stored outside the main project repo.
At a minimum, enable additional tests folders to be specified in the wscript using relative paths from the project root, as in:
This would allow processing of test texts that cannot be included in a public repo for various reasons (e.g. license restrictions). The test results would be placed in the normal results folder.
An additional option would be to allow folders to also be specified on the command line. These would not override the EXTRATESTS specified in the wscript, but rather be run in addition to them.