search

hercules-390 / hyperion

Hercules 390
Other
252 stars 68 forks source link

MISSING: "runtest" regression tests for PCC, CUnn and CSST instructions #106

Open Fish-Git opened 8 years ago

Fish-Git commented 8 years ago

There were some fixes to some instructions committed recently which are missing test cases for our "runtest" testing harness:

Runtest tests should eventually be created for each of them.

Runtest itself should probably be "formally" documented somewhere in our Hyperion Wiki as well.

Fish-Git commented 8 years ago

Jürgen,

Since you were the author of the above fixes, do you think you could put together a small test case for each to ensure each instruction is working properly (with respect to the problem you fixed in each)?

It doesn't have to be exact. It doesn't need to compile or /assemble. Feel free to use pseudo code if you wish.

Once we have something to start with, I'm sure I or someone else can then convert it into a formal runtest test case.

Thanks!

Fish-Git commented 8 years ago

Jurgen,

I have finished the Wiki page on runtest.

The URL of our Wiki is https://github.com/hercules-390/hyperion/wiki.

The URL of the RUNTEST page is https://github.com/hercules-390/hyperion/wiki/RUNTEST.

Briefly, all you do is create a Hercules .rc (run commands) file, but with a .tst file extension, which contains a special runtest command followed by a series of special comments preceding storage display commands.

The special comments indicate the storage values that you expect to see when your test completes.

The test is automatically run within Hercules by the runtest bash script (or runtest.cmd batch file on Windows) which post-processes the resulting Hercules log file through another script called redtest.rexx which then displays the results.

It may seem weird at first but it's really fairly straightforward. I encourage you to give it a try.

It's important that we have some test cases to verify the instructions we find bugs in have now been properly fixed and are working correctly, and to make sure we don't accidentally break things again in the future.

I would try doing it myself but I'm not sure I understand how the instructions that you fixed actually work! Since you found the bug and implemented the fix you already know the instructions better than anyone else and so could probably do a much better job than anyone else!

Thanks.

(And take your time! We know you're busy! I just don't want you to forget about it completely! Thanks!)

Juergen-Git commented 8 years ago

Hi Fish

sorry for the late reply. I only now saw your messages, because they didn't get mailed to me. Probably you wrote this shortly before I signed up to Github and the hyperion repository.

Concerning the test cases: I'm assuming, John is already at creating them, as in one of his replies around my real iron tests he wrote:

"However, this is only a test of a few of the function codes these instructions support. Building comprehensive test cases is going to be rather non-trivial, but at least I have an oracle; thanks for getting me started."

So, I think, I implicitly did what you wanted: Providing a starting point for John ;-)

Cheers Jürgen