jtrts should report an error if john ends with $? != 0, e.g., after a core dump.

[frank-dittrich]

See https://github.com/magnumripper/JohnTheRipper/issues/825 as an example.

[jfoug]

How does this look?

form=dynamic_54 guesses: 1500 0:00:00:00 DONE [PASSED] .pot CHK:dynamic_54 guesses: 1500 0:00:00:00 DONE [PASSED]

form=dynamic_55 guesses: 1500 0:00:00:04 DONE [PASSED] .pot CHK:dynamic_55 guesses: 1500 0:00:00:01 DONE [PASSED] * Error in `../run/john': munmap_chunk(): invalid pointer: 0x00007fa14e807010 *

form=dynamic_56 guesses: 1500 0:00:00:05 DONE [pass, but return code 34304] .pot CHK:dynamic_56 guesses: 1500 0:00:00:01 DONE [PASSED]

form=dynamic_57 guesses: 1500 0:00:00:06 DONE [PASSED] .pot CHK:dynamic_57 guesses: 1500 0:00:00:01 DONE [PASSED] .... form=dynamic_65 guesses: 1500 0:00:00:04 DONE [PASSED] .pot CHK:dynamic_65 guesses: 1500 0:00:00:01 DONE [PASSED] * Error in `../run/john': munmap_chunk(): invalid pointer: 0x00007fb0a0ee7010 *

form=dynamic_66 guesses: 1500 0:00:00:05 DONE [pass, but return code 34304] .pot CHK:dynamic_66 guesses: 1500 0:00:00:01 DONE [PASSED]

[magnumripper]

Nice :+1:

[jfoug]

f4f6fe7 Only checks original run and .pot re-run for error codes. We might want to add some logic to the 'run-one' function (prelims). It is not added to the -show runs which get counts.

[frank-dittrich]

(master)test $ ./jtrts.pl -type sapb -noprelims   
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.12.16, Sept 24, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper password cracker, version 1.8.0.2-bleeding-jumbo_omp [linux-gnu 64-bit AVX-autoconf]
--------------------------------------------------------------------------------

John Jumbo build detected.

form=sapB                         guesses: 1500 0:00:00:00 DONE  [pass, but return code 256]
.pot CHK:sapB                     guesses: 1500 0:00:00:00 DONE  [pass, but return code 256]

All tests passed without error.  Performed 1 tests.  Time used was 0 seconds
(master)test $ echo $?
0
(master)test $ ./jtrts.pl -type sapb -noprelims -q
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.12.16, Sept 24, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper password cracker, version 1.8.0.2-bleeding-jumbo_omp [linux-gnu 64-bit AVX-autoconf]
--------------------------------------------------------------------------------
All tests passed without error.  Performed 1 tests.  Time used was 0 seconds                
(master)test $ echo $?
0

I think, -q shouldn't suppress the pass, but return code ... warnings. Also, the number of such warnings should be printed in the summary.

Furthermore, ./jtrts.pl should probably finish with an exit code != 0 if one of the tests failed.

[jfoug]

Please test 5a09800

This should always show non-zeros (even in -q). It should also do a summary of them. It now shows them on !!!FAIL!!! also.

-no-prelim mode is now DEFAULT. TO get prelims, you have to add a -prelim switch. NOTE, this was helpful at one time, but I think enough people complain about it, to remove it.

[magnumripper]

-no-prelim mode is now DEFAULT

Excellent!

[jfoug]

Furthermore, ./jtrts.pl should probably finish with an exit code != 0 if one of the tests failed.

b4a1fcc changed the return code to be non-zero if there are any test problems on the run.