afd
commented
1 month ago Hi Jonathan
You are right that at present the output checking would support multi-line output. However, the checking for mutant output is geared towards one line of output per mutant, so I think it's best if we stick with tests that produce just one line of output.
Feel to do a PR adding better checks for this in the Python file if you like - it would be better if a test that produces multiple lines of output (when no mutants are enabled) would fail.
Thanks
Ally
From: JFJJ @.>
Sent: 08 July 2024 18:32
To: mc-imperial/dredd @.>
Cc: Donaldson, Alastair F @.>; State change @.>
Subject: Re: [mc-imperial/dredd] Add execute tests (PR #255)
@JonathanFoo0523 commented on this pull request.
In test/execute/README.mdhttps://github.com/mc-imperial/dredd/pull/255#discussion_r1669028872:
@@ -0,0 +1,20 @@ +Dredd execute tests +===================== + +There is one subdirectory per test. + +Each subdirectory must contain: + +-
harness.c(c)- a C or C++ file with amain, that should print a single line of output. + +-tomutate.c(c)- a C or C++ file that will be mutated. The harness should call functions of this file. + +-original.txt- A file containing the single line of output that should be printed by the harness when no mutants are enabled.
Moreover, the expected output is extracted by
open(test_directory / 'original.txt', 'r').read()
which should read more than a line.
— Reply to this email directly, view it on GitHubhttps://github.com/mc-imperial/dredd/pull/255#discussion_r1669028872, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABEFEXKSS5ZESH22LMPVFMTZLLEMPAVCNFSM6AAAAABKQZ7DKCVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDCNRTHE3DIOBWGY. You are receiving this because you modified the open/close state.Message ID: @.***>
Adds infrustructure and one example of a new kind of
executetest, that allows checking of the various program outputs that should be expected when all mutations for a file are considered.Fixes #254.