Also, the harness looks for two environment variables:
The tremendous amount of debug info now only shows up if you have CON4M_DEV set (ignores value).
The default search location is pulled from CON4M_TEST_DIR if provided.
The test specification goes in the SECOND of the two doc strings in a con4m file. If you provide an explicit file name and it does not have a second doc string, it is compiled and run, but not treated as a test case.
The test runner currently allows two possible items in the test spec (both may exist):
$output:
Matches the basic output from the builtin print. The builtin print is meant to be temporary while developing though; once I have kw args done in the language, I'll re-implement it. And I also will, at that point, change the runner to use my subprocess library.
$errors:
Matches against an ordered list of expected error codes.
$errors: is always checked, and if you leave it out, the runner assumes it should see no errors/warnings.
$output: is also optional; if you leave it out, the output will not be checked. Eventually I may add some pattern matching here instead. But it's good enough for now.
For tests that don't output, but that should be checked (i.e., they output error info if there's an error), you can specify $output: with nothing after it.
The output and the contents of each of the two spec items get stripped before comparison.
When you run
dev test
, if you do not provide an argument, it assumes that it should run over the entire tests directory.If you do provide arguments, you can provide a list of files or directories.
When you specify a directory, it will be selective about what it runs:
As part of supporting this, I added:
The former returns one of these constants:
Also, the harness looks for two environment variables:
CON4M_DEV
set (ignores value).CON4M_TEST_DIR
if provided.The test specification goes in the SECOND of the two doc strings in a con4m file. If you provide an explicit file name and it does not have a second doc string, it is compiled and run, but not treated as a test case.
The test runner currently allows two possible items in the test spec (both may exist):
$output:
Matches the basic output from the builtinprint
. The builtin print is meant to be temporary while developing though; once I have kw args done in the language, I'll re-implement it. And I also will, at that point, change the runner to use my subprocess library.$errors:
Matches against an ordered list of expected error codes.$errors:
is always checked, and if you leave it out, the runner assumes it should see no errors/warnings.$output:
is also optional; if you leave it out, the output will not be checked. Eventually I may add some pattern matching here instead. But it's good enough for now.For tests that don't output, but that should be checked (i.e., they output error info if there's an error), you can specify
$output:
with nothing after it.The output and the contents of each of the two spec items get stripped before comparison.
Fixes: https://github.com/crashappsec/libcon4m/issues/37