thoni56 / jregr

Regression testing of program output to expected
1 stars 0 forks source link

Feature for platform independent 'content' and 'exists' #1

Open thoni56 opened 3 years ago

thoni56 commented 3 years ago

Sometimes a test want to do some things that a normal script could do, e.g.

$ cat file.out

to get the content of a file. Or

$ ls file.out

to ensure a file exists. (Actually you could do that in many other ways...)

This creates a couple of problems when considering platform independence.

  1. Not all commands exists on all platforms
  2. The exact output of a command might differ for the same result

E.g if you use ls file.out to get "There is no such file: file.out" on some other platform that message might look different, causing a false negative test.

In general it would be nice if Jregr had some built-in commands for common cases. The first could possibly be exists and content of.

thoni56 commented 3 years ago

As a first idea, the notion of $1 and $2 could be extended to include $exists which would cause Jregr itself to print either

file.out : Exists

or

file.out : Does not exist

Given that this would go into the output of the test, it would match when comparing to .expected on all platforms.

thoni56 commented 3 years ago

Another way to handle this would be to introduce a ? operator to be used instead of the :. It would serve as a conditional, and since it is conditional, the output from it should no go into the output. So

.ext ? rm $.tmp

would both do the command conditionally on the existance of <case>.ext but also not include any output of that command.

And also since an extension will be checked in many cases you can do

.ext : 

To get a text in the output if the file <case>.ext does not exist.

thoni56 commented 3 years ago

The optional comand and empty command is implemented in 0.2.0 so that removes at least the need for $exists and differing output from shell commands.