Closed mjambon closed 8 months ago
This was already mostly supported in the form of output masking as long as the output can be converted into a string to be compared with a reference.
The masking/normalization method is not applicable for comparing e.g. a float against an other float within a margin of error such as fun old new_ -> abs_float (new_ -. old) < 1.
.
The remark above makes me wonder if we should rename mask_output
with normalize_output
which reflects its generic nature better.
The current mechanism for checking stdout or stderr compares the output of the test with previously captured output stored in the
snapshots
folder. This is fine but sometimes the output is variable due to timestamps or temporary file paths that differ from one run to another. This problem is mostly solved by masking the variable parts of the output before the comparison. However, this gets impractical when we're only looking for a specific substring or pattern in the output and we don't care about the rest. To address this problem, I suggest adding an option to customize the comparison of captured output:This would be an extra option of the
Testo.create
function.Diff
would be the default, built-in way to compare captured output. Creating a test would look like this: