openwall / john-tests

Test Suite for John the Ripper
24 stars 15 forks source link

TS errors after new Unicode defaults #52

Closed frank-dittrich closed 9 years ago

frank-dittrich commented 9 years ago

Not sure yet whether this is due to the recent memdbg, utf-8 default, or other changes, or a mix of those.


$ ./jtrts.pl -q
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper password cracker, version 1.8.0.4-jumbo-1-bleeding_omp_memdbg [linux-gnu 64-bit AVX-autoconf]
--------------------------------------------------------------------------------
Warning: SAP-B format should never be UTF-8.
Use --target-encoding=iso-8859-1 or whatever is applicable.
form=dynamic_29-raw               guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=dynamic_29                   guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=dynamic_33                   guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=dynamic_33                   guesses: 1327 -show=1327 0:00:00:00 DONE : Expected count(s) (1500)(1365)  [!!!FAILED1!!!]
form=dynamic_33-NT                guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=dynamic_1032                 guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:dynamic_1032             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=dynamic_1400                 guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:dynamic_1400             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=dynamic_1400                 guesses:    0 -show=   0 unk unk : Expected count(s) (1500)(920)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:dynamic_1400             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=raw-md5u-raw                 guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=raw-md5u                     guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=raw-md5u                     guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=raw-md5u-raw                 guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=NT                           guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=NT-raw                       guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=mscash                       guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=mscash_crash                 guesses: 1169 -show=1169 0:00:00:00 DONE : Expected count(s) (1440)  [!!!FAILED1!!!]
form=pwdump_lm                    guesses:  646 -show=2385 0:00:00:00 DONE : Expected count(s) (986)(-show2760)  [!!!FAILED1!!!]
form=pwdump_nt                    guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=krb5pa-md5                   guesses: 1229 -show=1229 0:00:00:02 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=krb5pa-md5                   guesses: 1442 -show=1442 0:00:00:01 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=MSSql12                      guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=MSSql05                      guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=MSSql                        guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
.pot CHK:MSSql-8859-1             guesses: 1195 -show=1195 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1195 val-pwd  0 inval-pwd)
.pot CHK:MSSql-8859-1-gen         guesses:  390 -show= 390 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (390 val-pwd  0 inval-pwd)
form=oracle                       guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=netntlm-naive                guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=netntlm-naive-l0phtcrack     guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=netlm-l0phtcrack             guesses: 1069 -show=1069 0:00:00:00 DONE : Expected count(s) (1351)  [!!!FAILED1!!!]
form=netlmv2                      guesses: 1229 -show=1229 0:00:00:02 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=netntlmv2                    guesses: 1229 -show=1229 0:00:00:02 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=mschapv2-naive               guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=episerver_sha1               guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=episerver_sha256             guesses: 1229 -show=1229 0:00:00:01 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
sh: line 1:  3639 Segmentation fault      (core dumped) ../run/john -ses=tst -pot=tst.pot sybasease_tst.in --wordlist=pw.dic -form=sybasease 2>&1 > /dev/null
form=sybasease                    guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 139]
.pot CHK:sybasease                guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
[...]
frank-dittrich commented 9 years ago

All formats pass self test, though.

frank-dittrich commented 9 years ago

Most errors occur for non-memdbg builds as well, will post a second list in a while. Afterwards, I'll try reverting the utf-8 default change.

frank-dittrich commented 9 years ago

Now without memdbg:

-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper password cracker, version 1.8.0.4-jumbo-1-bleeding_omp [linux-gnu 64-bit AVX-autoconf]
--------------------------------------------------------------------------------
Warning: SAP-B format should never be UTF-8.
Use --target-encoding=iso-8859-1 or whatever is applicable.
form=dynamic_29-raw               guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=dynamic_29                   guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=dynamic_33                   guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=dynamic_33                   guesses: 1327 -show=1327 0:00:00:00 DONE : Expected count(s) (1500)(1365)  [!!!FAILED1!!!]
form=dynamic_33-NT                guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=dynamic_1032                 guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:dynamic_1032             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=dynamic_1400                 guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:dynamic_1400             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=dynamic_1400                 guesses:    0 -show=   0 unk unk : Expected count(s) (1500)(920)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:dynamic_1400             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=raw-md5u-raw                 guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=raw-md5u                     guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=raw-md5u                     guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=raw-md5u-raw                 guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=NT                           guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=NT-raw                       guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=mscash                       guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=mscash_crash                 guesses: 1169 -show=1169 0:00:00:00 DONE : Expected count(s) (1440)  [!!!FAILED1!!!]
form=pwdump_lm                    guesses:  646 -show=2385 0:00:00:00 DONE : Expected count(s) (986)(-show2760)  [!!!FAILED1!!!]
form=pwdump_nt                    guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=krb5pa-md5                   guesses: 1229 -show=1229 0:00:00:04 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=krb5pa-md5                   guesses: 1442 -show=1442 0:00:00:01 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=MSSql12                      guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=MSSql05                      guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=MSSql                        guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
.pot CHK:MSSql-8859-1             guesses: 1195 -show=1195 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1195 val-pwd  0 inval-pwd)
.pot CHK:MSSql-8859-1-gen         guesses:  390 -show= 390 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (390 val-pwd  0 inval-pwd)
form=oracle                       guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=netntlm-naive                guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=netntlm-naive-l0phtcrack     guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=netlm-l0phtcrack             guesses: 1069 -show=1069 0:00:00:00 DONE : Expected count(s) (1351)  [!!!FAILED1!!!]
form=netlmv2                      guesses: 1229 -show=1229 0:00:00:02 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=netntlmv2                    guesses: 1229 -show=1229 0:00:00:02 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=mschapv2-naive               guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=episerver_sha1               guesses: 1229 -show=1229 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=episerver_sha256             guesses: 1229 -show=1229 0:00:00:01 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
sh: line 1: 13726 Segmentation fault      (core dumped) ../run/john -ses=tst -pot=tst.pot sybasease_tst.in --wordlist=pw.dic -form=sybasease 2>&1 > /dev/null
form=sybasease                    guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 139]
.pot CHK:sybasease                guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=office_2007                  guesses: 1190 -show=1190 0:00:00:01 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=office_2010                  guesses:   32 -show=  32 0:00:00:02 DONE : Expected count(s) (48)  [!!!FAILED1!!!]
form=office_2013                  guesses:   32 -show=  32 0:00:00:07 DONE : Expected count(s) (48)  [!!!FAILED1!!!]
.pot CHK:mschapv2-naive-koi8r     guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:netlmv2-koi8r            guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:netntlm-naive-koi8r      guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:netntlm-naive(l0phtcrack guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:netntlmv2-koi8r          guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:NT-koi8r                 guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:mscash-koi8r             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:mssql-koi8r-gen          guesses:  385 -show= 385 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (385 val-pwd  0 inval-pwd)
.pot CHK:mssql-koi8r              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:mssql05-koi8r-gen        guesses:  385 -show= 385 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (385 val-pwd  0 inval-pwd)
.pot CHK:MSSql05-koi8r            guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:oracle-koi8r             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
Some tests had Errors. Performed 429 tests.  36 errors  18 errors reprocessing the .POT files  4 runs had non-clean exit
Time used was 925 seconds
frank-dittrich commented 9 years ago

Most of the errors disappear after reverting JtR commit 90e21c5bae90258f7b6420cda8cc4a72b725c8ac (default to utf-8), so the test suite might need to specify encoding now (for jumbo versions).

magnumripper commented 9 years ago

Oh right, the test suite is not aware of the new defaults. Most problems should go aways with -pass=-enc=raw but there may be some tweaks needed in TS.

jfoug commented 9 years ago

Adding "--encoding=raw" to the extra input on dyna_29-raw gets 1500 passwords cracked. BUT now on the .pot re-run (which creates the passwords from the .pot file), only 1195 get cracked. I have tried adding --target-encoding=raw but that changes nothing. I am not sure which direction to head

magnumripper commented 9 years ago

One option discussed before is using a testsuite.conf instead. I don't care about backwards Jumbo compatibility in TS, but we do need to handle core john - so we'd need to do this selectively.

But then we'd not test an actual default run. To do that we probably need to change the tests.

magnumripper commented 9 years ago

Adding "--encoding=raw" to the extra input on dyna_29-raw gets 1500 passwords cracked. BUT now on the .pot re-run (which creates the passwords from the .pot file), only 1195 get cracked. I have tried adding --target-encoding=raw but that changes nothing.

To actually dissect this, here's what happens: Because of UnicodeStoreUTF8 = Y and CPstoreUTF8 = Y, the cracked 8-bit raw password end up as UTF-8 in the pot file. So for the .pot re-run you should probably use -input-enc=utf8 -target-enc=raw.

jfoug commented 9 years ago

To have different 'extra' params for .pot re-check will require code changes to the perl script, and a new layout in the .dat file (adding additional params).

jfoug commented 9 years ago

Rerunning the failed test using your suggestion:

../run/john -ses=tst  -pot=tst.pot tst.in --wordlist=pw3 -form=dynamic_29 --encoding=utf8 --target-encoding=raw

gives us 1500 cracks when re-running the data from the .pot file (words were cut out and put into pw3 by the perl script).

jfoug commented 9 years ago

Because of UnicodeStoreUTF8 = Y and CPstoreUTF8 = Y, the cracked 8-bit raw password end up as UTF-8 in the pot file.

Ok, why did the --target-encoding=raw not handle this situation? I would think that the command line should take precedence here no matter what was in the .conf file. I tried with both --encoding=raw and --target-encoding=raw and I received the same results (1195 on .pot check).

magnumripper commented 9 years ago

Ok, why did the --encode-target=raw not handle this situation? I would think that the command line should take precedence here no matter what was in the .conf file.

Because these are not target encodings, they are john.pot encodings. We currently don't have any option for overriding them.

magnumripper commented 9 years ago

To have different 'extra' params for .pot re-check will require code changes to the perl script, and a new layout in the .dat file (adding additional params).

The current "-enc=xxx" option can be used I think. Or rather the lack of it: If there is no "-enc" then we should be able to use "-enc=raw" and get away with it.

jfoug commented 9 years ago

I have worked through some things in the TS. BUT I think the changes in going utf-8 .pot file may have some issues for some formats using encoding.

I made these changes in jtrts.pl: [x] handle a dual field (the additionalParams field, #9). It can now have a | char and if so, the data after the | is for the .pot re-check. [x] added a ExtraArgs_RunPot() function. This grabs the 2nd additionalParams data if it exists. Then if no -enc is found, this function adds -enc=utf8 [x] the .pot recheck command string is now created. It is not a copy of the normal run, where we replace strings, like it was before.

In the jtrts.dat file, I have added -enc=raw to numerous formats (and other changes).

I am still testing. However, I am seeing some cases where what is written into the .pot file may not be usable re-input back into jtr.

jfoug commented 9 years ago

Here is where I am at now (normal jumbo run, not a full run) 43393c1

$ ./jtrts.pl -q
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper password cracker, version 1.8.0.4-jumbo-1-bleeding_omp [cygwin 64-bit AVX-autoconf]
--------------------------------------------------------------------------------
Warning: SAP-B format should never be UTF-8.
Use --target-encoding=iso-8859-1 or whatever is applicable.
OpenCL error (CL_INVALID_MEM_OBJECT) in file (opencl_sha1crypt_fmt_plug.c) at line (165) - (Release mem in)
.pot CHK:krb5pa-md5-new           guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:MSSql-8859-1             guesses: 1195 -show=1195 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1195 val-pwd  0 inval-pwd)
.pot CHK:MSSql-8859-1-gen         guesses:  390 -show= 390 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (390 val-pwd  0 inval-pwd)
.pot CHK:oracle                   guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1243 val-pwd  257 inval-pwd)
Some tests had Errors. Performed 429 tests.  4 errors reprocessing the .POT files
Time used was 799 seconds
jfoug commented 9 years ago

After a few changes 4d59970, here is the results for a 'full' run.

.pot CHK:krb5pa-md5-new           guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:MSSql-8859-1             guesses: 1195 -show=1195 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1195 val-pwd  0 inval-pwd)
.pot CHK:MSSql-8859-1-gen         guesses:  390 -show= 390 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (390 val-pwd  0 inval-pwd)
.pot CHK:oracle                   guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1243 val-pwd  257 inval-pwd)
.pot CHK:mssql-cp850              guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1471 val-pwd  29 inval-pwd)
.pot CHK:oracle-cp850             guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1418 val-pwd  82 inval-pwd)
.pot CHK:mssql-cp858              guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1471 val-pwd  29 inval-pwd)
.pot CHK:oracle-cp858             guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1436 val-pwd  64 inval-pwd)
.pot CHK:mssql-cp437              guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1185 val-pwd  315 inval-pwd)
.pot CHK:oracle-cp437             guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1154 val-pwd  346 inval-pwd)
.pot CHK:mschapv2-iso8859-15      guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:netntlm-iso8859-15       guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:netntlm(l0phtcrack)-iso8 guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:NT-iso8859-15            guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:mscash-iso8859-15        guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:mssql-cp1252             guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1472 val-pwd  28 inval-pwd)
.pot CHK:oracle-cp1252            guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1444 val-pwd  56 inval-pwd)
jfoug commented 9 years ago

I am seeing more .pot errors on my linux builds (and sparc), than I see on cygwin. I am not sure why.

frank-dittrich commented 9 years ago

The latest commit makes testing of core fail:

(master)test $ ./jtrts.pl 
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper password cracker, version 1.8.0.4
--------------------------------------------------------------------------------

John CORE build detected.  Only core formats can be tested.

form=md5crypt                     guesses: 1500 0:00:00:12 DONE  [PASSED]
.pot CHK:md5crypt                 guesses: 1500 0:00:00:09 100%  [PASSED] (1500 val-pwd)

form=md5crypt                     guesses: 1500 0:00:00:01 DONE  [PASSED]
.pot CHK:md5crypt                 guesses: 1500 0:00:00:00 100%  [PASSED] (1500 val-pwd)

form=md5_apr1                     guesses: 1500 0:00:00:12 DONE  [PASSED]
.pot CHK:md5_apr1                 guesses: 1500 0:00:00:10 100%  [PASSED] (1500 val-pwd)

form=md5_apr1                     guesses: 1500 0:00:00:01 DONE  [PASSED]
.pot CHK:md5_apr1                 guesses: 1500 0:00:00:00 100%  [PASSED] (1500 val-pwd)

form=BSDIcrypt                    guesses: 1500 0:00:00:01 DONE  [PASSED]
.pot CHK:BSDIcrypt                guesses: 1500 0:00:00:01 100%  [PASSED] (1500 val-pwd)

form=Bcrypt                       guesses: 1500 0:00:00:02 DONE  [PASSED]
.pot CHK:Bcrypt                   guesses: 1500 0:00:00:02 100%  [PASSED] (1500 val-pwd)

form=Bcrypt_broken                guesses: 1500 0:00:00:02 DONE  [PASSED]
.pot CHK:Bcrypt_broken            guesses: 1500 0:00:00:02 100%  [PASSED] (1500 val-pwd)

form=descrypt                     guesses: 1500 0:00:00:00 DONE  [PASSED]
.pot CHK:descrypt                 guesses: 1500 0:00:00:00 100%  [PASSED] (1500 val-pwd)

form=descrypt                     guesses: 1500 0:00:00:01 DONE  [PASSED]
.pot CHK:descrypt                 guesses: 1500 0:00:00:00 100%  [PASSED] (1500 val-pwd)

form=lm                           guesses: 3000 0:00:00:00 DONE  [PASSED]
.pot CHK:lm                       guesses: 1500 0:00:00:00 100%  [PASSED] (1500 val-pwd)

form=pwdump_lm                    guesses:    0 -show=   0 unk unk : Expected count(s) (986)(-show2760)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:pwdump_lm                guesses:    0 unk unk  [pass, but exited, return code 1]

Some tests had Errors. Performed 11 tests.  1 errors  2 runs had non-clean exit
Time used was 65 seconds
jfoug commented 9 years ago

1cb2321 fixes core after recent changes.

jfoug commented 9 years ago

It appears to me, that the .pot re-checks (many at least) are where the length of the string is too long when converted into utf8, thus when trying to use the passwords from the .pot file, they are simply too long.

jfoug commented 9 years ago

Ok, here is the 'bug' (if we want to call it that). I think this will only affect UTF-8 formats.

A utf-8 input string that is <= PLAINTEXT_LENGTH in 'real' characters is > PLAINTEXT_LENGTH in utf-8 encoding. Within wordlist we do not decode this string, and simply truncate.

I am not sure how to work around this. The code in wordlist.c simply truncates like this line[length] = 0; What we really need is something like utf8_trunc(line, length); where that function will put a null at the proper location within the string. BUT I think that should only be done if the input encoding is utf8, and the format is a utf8 format. I really do not understand the encoding code all that well, is this logic right?

magnumripper commented 9 years ago

I see... but there's no way we can fix the truncation in wordlist.c - all of that is correct coded but limited by PLAINTEXT_BUFFER_SIZE.

I think there is only one way to solve this, but it's basically very simple: When TS creates the new dictionary from the .pot file, it has to decode the UTF-8 and again encode it into what was used as target encoding. And then the .pot recheck would again use -enc=raw and not -enc=utf8.

jfoug commented 9 years ago

That is a bad hack.

The .pot check was added 100% to make SURE that the exact data written to the .pot file actually was the right thing. It 'may' be, but now JtR can not check it out in certain cases. I guess that is just the way it will be.

I can work around this, if we KNOW this is the behavior on certain formats, by adding this amount into the .dat file for the .pot recheck count.

magnumripper commented 9 years ago

In a way it actually ensures functionality. Say we crack MÜLLER in CP850, it gets written to the pot file supposedly as UTF-8. Then we read it from .pot as UTF-8, convert it to CP850 and ensure it again gets cracked using -target-enc=cp850. If this works we actually confirmed proper conversions back and forth.

frank-dittrich commented 9 years ago

Just to mentiom the current status (non-GPU, 64bit Linux AVX):

 (master)test $ ./jtrts.pl -q
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper password cracker, version 1.8.0.6-jumbo-1-bleeding_omp [linux-gnu 64-bit AVX-autoconf]
--------------------------------------------------------------------------------
Warning: SAP-B format should never be UTF-8.
Use --target-encoding=iso-8859-1 or whatever is applicable.
.pot CHK:krb5pa-md5-new           guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:MSSql-8859-1             guesses: 1195 -show=1195 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1195 val-pwd  0 inval-pwd)
.pot CHK:MSSql-8859-1-gen         guesses:  390 -show= 390 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (390 val-pwd  0 inval-pwd)
.pot CHK:oracle                   guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1243 val-pwd  257 inval-pwd)
Some tests had Errors. Performed 431 tests.  4 errors reprocessing the .POT files       
Time used was 882 seconds

Not sure which of these are related to the utf-8 conversion.

(master)test $ ./jtrts.pl -v -stoponerror -type=krb5pa-md5
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper password cracker, version 1.8.0.6-jumbo-1-bleeding_omp [linux-gnu 64-bit AVX-autoconf]
--------------------------------------------------------------------------------

John Jumbo build detected.
Warning: SAP-B format should never be UTF-8.
Use --target-encoding=iso-8859-1 or whatever is applicable.
--pot=NAME option is valid
--encoding=NAME option is valid
all.chr (../run/all.chr) not found
alnum.chr found, inc_alnum_ee8763c850dee8e4b88ef547a8ed39b8 added as a capability
Types to filter on:
krb5pa-md5
Capabilities in this build of john:
jumbo core inc local_pot_valid encode_valid utf8 cp1252 cp1251 koi8r cp437 cp737 cp850 cp858 cp866 iso8859-1 iso8859-15 inc_alnum_ee8763c850dee8e4b88ef547a8ed39b8

form=krb5pa-md5                   guesses: 1500 0:00:00:02 DONE  [PASSED]
.pot CHK:krb5pa-md5               guesses: 1500 0:00:00:00 DONE  [PASSED] (1500 val-pwd)

form=krb5pa-md5-new               guesses: 1500 0:00:00:01 DONE  [PASSED]
.pot CHK:krb5pa-md5-new           guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
Exiting on error. The .pot file tst.pot contains the found data
The command used to run this test was:

../run/john -ses=tst  -pot=tst.pot krb5pa-md5_new_tst.in --wordlist=pw-new.dic -enc=raw

However, when I try to reproduce this error, I always get

(master)test $ rm tst.pot; rm tst.log; ../run/john -ses=tst  -pot=tst.pot krb5pa-md5_new_tst.in --wordlist=pw-new.dic -enc=raw > /dev/null ; wc -l tst.pot ; sort -u tst.pot|wc -l
Will run 8 OpenMP threads
Press 'q' or Ctrl-C to abort, almost any other key for status
1500g 0:00:00:01 DONE (2015-06-19 14:39) 1119g/s 2900p/s 4350Kc/s 4350KC/s aab2b..ko}GpppVeLj
Use the "--show" option to display all of the cracked passwords reliably
Session completed
1500 tst.pot
1500

Same for mssql:

(master)test $ ./jtrts.pl -v -stoponerror -type=MSSql
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper password cracker, version 1.8.0.6-jumbo-1-bleeding_omp [linux-gnu 64-bit AVX-autoconf]
--------------------------------------------------------------------------------

John Jumbo build detected.
Warning: SAP-B format should never be UTF-8.
Use --target-encoding=iso-8859-1 or whatever is applicable.
--pot=NAME option is valid
--encoding=NAME option is valid
all.chr (../run/all.chr) not found
alnum.chr found, inc_alnum_ee8763c850dee8e4b88ef547a8ed39b8 added as a capability
Types to filter on:
mssql
Capabilities in this build of john:
jumbo core inc local_pot_valid encode_valid utf8 cp1252 cp1251 koi8r cp437 cp737 cp850 cp858 cp866 iso8859-1 iso8859-15 inc_alnum_ee8763c850dee8e4b88ef547a8ed39b8

form=MSSql                        guesses: 1500 0:00:00:00 DONE  [PASSED]
.pot CHK:MSSql                    guesses: 1500 0:00:00:00 DONE  [PASSED] (1500 val-pwd)

form=MSSql-8859-1                 guesses: 1500 0:00:00:00 DONE  [PASSED]
.pot CHK:MSSql-8859-1             guesses: 1195 -show=1195 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1195 val-pwd  0 inval-pwd)
Exiting on error. The .pot file tst.pot contains the found data
The command used to run this test was:

../run/john -ses=tst  -pot=tst.pot mssql_tst.in --wordlist=pw.dic -enc=iso-8859-1
(master)test $ rm tst.pot; rm tst.log; ../run/john -ses=tst  -pot=tst.pot mssql_tst.in --wordlist=pw.dic -enc=iso-8859-1 > /dev/null ; wc -l tst.pot ; sort -u tst.pot|wc -l 
Warning: detected hash type "mssql", but the string is also recognized as "mssql05"
Use the "--format=mssql05" option to force loading these as that type instead
Warning: no OpenMP support for this hash type, consider --fork=8
Press 'q' or Ctrl-C to abort, almost any other key for status
1500g 0:00:00:00 DONE (2015-06-19 14:42) 50000g/s 54133p/s 54133c/s 41275KC/s 1__19..HELLO__19
Use the "--show" option to display all of the cracked passwords reliably
Session completed
1500 tst.pot
1500

So, is jtrts.pl really showing the exact command that caused only 1498 of 1500 passwords to be found?

frank-dittrich commented 9 years ago

Currently the test suite seems to be rather useless, but the segfaults are definitely john issues. Not sure whether it is still a jtrts issue or whether we have old / new john issues with latest jtrts and latest bleeding-jumbo:

(master)test $ ./jtrts.pl -type full -q -passthru="--skip-self-test"
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper 1.8.0.6-jumbo-1-195-g4fe4857 OMP [linux-gnu 64-bit AVX2-autoconf]
--------------------------------------------------------------------------------
Warning: SAP-B format should never be UTF-8.
Use --target-encoding=iso-8859-1 or whatever is applicable.
.pot CHK:dynamic_2                guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2                guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_3                guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_3                guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_12               guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED6!!! exited, return code 1]
.pot CHK:dynamic_12               guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED6!!! exited, return code 1]
.pot CHK:dynamic_13               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_13               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_15               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_15               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_16               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_16               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_17               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_17               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_18               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_18               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_24               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_24               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_25               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_25               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_26               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_26               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_34               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_34               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_35               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_35               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_36               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_36               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_37               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_37               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
sh: line 1:  1574 Segmentation fault      (core dumped) ../run/john -ses=tst --skip-self-test -pot=tst.pot tst.in --wordlist=pw3 2> _stderr
.pot CHK:dynamic_38               guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED6!!! exited, return code 139]
sh: line 1:  1626 Segmentation fault      (core dumped) ../run/john -ses=tst --skip-self-test -pot=tst.pot tst.in --wordlist=pw3 2> _stderr
.pot CHK:dynamic_38               guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED6!!! exited, return code 139]
.pot CHK:dynamic_39               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_39               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_40               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_40               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_50               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_50               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_60               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_60               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_61               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_61               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_70               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_70               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_80               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_80               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_90               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_90               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_100              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_100              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_110              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_110              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_120              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_120              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_130              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_130              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_140              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_140              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_150              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_150              guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1001             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1001             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1002             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1002             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1003             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1003             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1004             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1004             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1005             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1005             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1006             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1006             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1010             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED6!!! exited, return code 1]
.pot CHK:dynamic_1010             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED6!!! exited, return code 1]
.pot CHK:dynamic_1014             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1014             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1016             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1016             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1017             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1017             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1023             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1023             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1026             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1026             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1027             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1027             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1028             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1028             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1029             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1029             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1030             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1030             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1031             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1031             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1032             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)(1434)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1501             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1501             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1502             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1502             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1503             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED6!!! exited, return code 1]
.pot CHK:dynamic_1503             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED6!!! exited, return code 1]
.pot CHK:dynamic_1504             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1504             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2000             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2000             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2001             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2001             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2002             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2002             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2003             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2003             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2004             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2004             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2005             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2005             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2006             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2006             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2008             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2008             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2009             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2009             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
sh: line 1:  3144 Segmentation fault      (core dumped) ../run/john -ses=tst --skip-self-test -pot=tst.pot tst.in --wordlist=pw3 2> _stderr
.pot CHK:dynamic_2010             guesses:    0 -show=   0 0:00:00:14 DONE : Expected count(s) (1500)  [!!!FAILED6!!! exited, return code 139]
sh: line 1:  3194 Segmentation fault      (core dumped) ../run/john -ses=tst --skip-self-test -pot=tst.pot tst.in --wordlist=pw3 2> _stderr
.pot CHK:dynamic_2010             guesses:    0 -show=   0 0:00:00:14 DONE : Expected count(s) (1500)  [!!!FAILED6!!! exited, return code 139]
sh: line 1:  3207 Segmentation fault      (core dumped) ../run/john -ses=tst --skip-self-test -pot=tst.pot tst.in --wordlist=pw3 2> _stderr
.pot CHK:dynamic_2011             guesses:    0 -show=   0 0:00:00:14 DONE : Expected count(s) (1500)  [!!!FAILED6!!! exited, return code 139]
sh: line 1:  3261 Segmentation fault      (core dumped) ../run/john -ses=tst --skip-self-test -pot=tst.pot tst.in --wordlist=pw3 2> _stderr
(master)test $ ../run/john --list=build-info 
Version: 1.8.0.6-jumbo-1-195-g4fe4857
Build: linux-gnu 64-bit AVX2-autoconf OMP
SIMD: AVX2, interleaving: MD4:3 MD5:3 SHA1:1 SHA256:1 SHA512:1
$JOHN is ../run/
Format interface version: 13
Max. number of reported tunable costs: 3
Rec file version: REC4
Charset file version: CHR3
CHARSET_MIN: 1 (0x01)
CHARSET_MAX: 255 (0xff)
CHARSET_LENGTH: 24
Max. Markov mode level: 400
Max. Markov mode password length: 30
gcc version: 5.1.1
GNU libc version: 2.21 (loaded: 2.21)
Crypto library: OpenSSL
OpenSSL library version: 0100010bf
OpenSSL 1.0.1k-fips 8 Jan 2015
GMP library version: 6.0.0
File locking: fcntl()
fseek(): fseek
ftell(): ftell
fopen(): fopen
memmem(): System's
frank-dittrich commented 9 years ago

With bleeding-jumbo commit https://github.com/magnumripper/JohnTheRipper/commit/751fc454f90c8a3718c784d7bf1b00d947765e6bm the list of failing dynamic formats is a lot shorter:

(master)test $ ./jtrts.pl -type dynamic -q -passthru="--skip-self-test"
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper 1.8.0.6-jumbo-1-218-g751fc45 ASan [linux-gnu 64-bit AVX2-autoconf]
--------------------------------------------------------------------------------
Warning: SAP-B format should never be UTF-8.
Use --target-encoding=iso-8859-1 or whatever is applicable.
.pot CHK:dynamic_2                guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_2                guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_3                guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_3                guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_34               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_34               guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1001             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1001             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1002             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1002             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1004             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1004             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1005             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1005             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1006             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
.pot CHK:dynamic_1006             guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  0 inval-pwd)
Some tests had Errors. Performed 232 tests.  16 errors reprocessing the .POT files      
Time used was 807 seconds
frank-dittrich commented 9 years ago

Probably, my list from 3 days ago was incomplete (I copied just the failed dynamic tests), because these are the non-dynamic CPU formats that currently fail (ASan and UbSan enabled):

form=crc32                        guesses:    1 -show=   1 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=crc32_read_file              guesses:    1 -show=   1 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=crc32_dups                   guesses:    1 -show=   1 0:00:00:00 DONE : Expected count(s) (1500)(-show50)  [!!!FAILED1!!!]
form=crc32_dups2                  guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)(-show50)  [!!!FAILED1!!!]
.pot CHK:crc32_dups2              guesses:    0 -show=   0 unk unk : Expected count(s) (50)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=crc32_dups_read_file         guesses:    1 -show=   1 0:00:00:00 DONE : Expected count(s) (1500)(-show50)  [!!!FAILED1!!!]
form=crc32_dups2_read_file        guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)(-show50)  [!!!FAILED1!!!]
.pot CHK:crc32_dups2_read_file    guesses:    0 -show=   0 unk unk : Expected count(s) (50)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=crc32_dups_dupe_rem          guesses:    1 -show=   1 0:00:00:00 DONE : Expected count(s) (50)  [!!!FAILED1!!!]
form=crc32_dups2_dupe_rem         guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (50)  [!!!FAILED1!!!]
.pot CHK:crc32_dups2_dupe_rem     guesses:    0 -show=   0 unk unk : Expected count(s) (50)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
.pot CHK:krb5pa-md5-new           guesses: 1498 -show=1498 0:00:00:04 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:MSSql-8859-1             guesses: 1195 -show=1195 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1195 val-pwd  0 inval-pwd)
.pot CHK:MSSql-8859-1-gen         guesses:  390 -show= 390 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (390 val-pwd  0 inval-pwd)
.pot CHK:oracle                   guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1243 val-pwd  257 inval-pwd)
form=raw-keccak-512               guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:raw-keccak-512           guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=raw-keccak-256               guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:raw-keccak-256           guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
.pot CHK:mssql-cp850              guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1471 val-pwd  29 inval-pwd)
.pot CHK:oracle-cp850             guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1418 val-pwd  82 inval-pwd)
.pot CHK:mssql-cp858              guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1471 val-pwd  29 inval-pwd)
.pot CHK:oracle-cp858             guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1436 val-pwd  64 inval-pwd)
.pot CHK:mssql-cp437              guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1185 val-pwd  315 inval-pwd)
.pot CHK:oracle-cp437             guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1154 val-pwd  346 inval-pwd)
.pot CHK:mschapv2-iso8859-15      guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:netntlm-iso8859-15       guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:netntlm(l0phtcrack)-iso8 guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:NT-iso8859-15            guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:mscash-iso8859-15        guesses: 1498 -show=1498 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1498 val-pwd  0 inval-pwd)
.pot CHK:mssql-cp1252             guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1472 val-pwd  28 inval-pwd)
.pot CHK:oracle-cp1252            guesses: 1500 -show=1500 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED4!!!]  (1444 val-pwd  56 inval-pwd)
form=rar                          guesses:  216 -show= 216 0:00:05:57 DONE : Expected count(s) (297)  [!!!FAILED1!!!

the raw-keccak errors must be related to https://github.com/magnumripper/JohnTheRipper/issues/1492

jfoug commented 9 years ago

@frank-dittrich

Please have a look at the TS again. It still is having a couple of problems, but I think it is back to usable.

NOTE, the Crc32 problem is in john. I do not see any problems under cygwin, BUT it has something wrong when I run under either 32 or 64 bit Fedora. I have not looked into it, because I am not overly concerned with that format.

I AM seeing AVX2 problems on a 32 bit linux build. These are real problems with JtR, not the TS.

jfoug commented 9 years ago

@frank-dittrich and @magnumripper how is the TS working now, that I have a john-local.conf that undoes the unicode defaults? It is working fine for me, except for the crc32 function, which for some reason on my linux boxes, only cracks 1 or 2 hashes. The TS is showing it like it is, I can run that format by hand, and get the same results (1 or 2 hashes).

magnumripper commented 9 years ago
$ ./jtrts.pl crc32
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper 1.8.0.6-jumbo-1-565-gc0560a4 OMP [darwin14.4.0 64-bit AVX-ac]
--------------------------------------------------------------------------------

John Jumbo build detected.

form=crc32                        guesses: 1500 0:00:00:00 DONE  [PASSED]
.pot CHK:crc32                    guesses: 1500 0:00:00:00 DONE  [PASSED] (1500 val-pwd)

form=crc32_read_file              guesses: 1500 0:00:00:00 DONE  [PASSED]
.pot CHK:crc32_read_file          guesses: 1500 0:00:00:00 DONE  [PASSED] (1500 val-pwd)

form=crc32_dups                   guesses:   50 0:00:00:00 DONE  [PASSED]
.pot CHK:crc32_dups               guesses:   50 0:00:00:00 DONE  [PASSED] (50 val-pwd)

form=crc32_dups2                  guesses:   50 0:00:00:00 DONE  [PASSED]
.pot CHK:crc32_dups2              guesses:   50 0:00:00:00 DONE  [PASSED] (50 val-pwd)

form=crc32_dups_read_file         guesses:   50 0:00:00:00 DONE  [PASSED]
.pot CHK:crc32_dups_read_file     guesses:   50 0:00:00:00 DONE  [PASSED] (50 val-pwd)

form=crc32_dups2_read_file        guesses:   50 0:00:00:00 DONE  [PASSED]
.pot CHK:crc32_dups2_read_file    guesses:   50 0:00:00:00 DONE  [PASSED] (50 val-pwd)

form=crc32_dups_dupe_rem          guesses:   50 0:00:00:00 DONE  [PASSED]
.pot CHK:crc32_dups_dupe_rem      guesses:   50 0:00:00:00 DONE  [PASSED] (50 val-pwd)

form=crc32_dups2_dupe_rem         guesses:   50 0:00:00:00 DONE  [PASSED]
.pot CHK:crc32_dups2_dupe_rem     guesses:   50 0:00:00:00 DONE  [PASSED] (50 val-pwd)

All tests passed without error.  Performed 8 tests.  Time used was 56 seconds

No problem here

$ ./jtrts.pl -internal crc32
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper 1.8.0.6-jumbo-1-565-gc0560a4 OMP [darwin14.4.0 64-bit AVX-ac]
--------------------------------------------------------------------------------

John Jumbo build detected.
Running JTRTS in -internal mode

form=crc32                        guesses:    4 -show=   4 0:00:00:00 DONE : Expected count(s) (8)(-show8)  [!!!FAILED1!!!]
.pot CHK:crc32                    guesses:    4 0:00:00:00 DONE  [PASSED] (4 val-pwd)
Some tests had Errors. Performed 1 tests.  1 errors
Time used was 7 seconds

I think the above problems are from fmt_not_exact and/or dupe test vectors

magnumripper commented 9 years ago

I'm running a full test now

jfoug commented 9 years ago

Note, for a truely 'full' test you will need to do: ./jtrts.pl -dynamic=all -q full If you have already run the full test, then ./jtrts.pl -dynamic=all dynamic will do what was not done in the full test.

-dynamic=all dynamic (or full) will not work right until after the patch I did today. the -dynamic=all will add all 'hidden' dynamic into the available types, BUT on the run if there is no -format=dynamic_?? then john will not find any hashes since the format is 'hidden'. That part of the implementation of hidden I do not care for. I would like john to be able to load a file like that even with the format hidden, but it can not, because the format was not initialized at all and thus no format record to have a valid()

magnumripper commented 9 years ago

Output ruined by lots of Use of uninitialized value in split at ./jtrts.pl line 718

jfoug commented 9 years ago

I will fix that. I bet this was due to running 'full' and not running '-dynamic-all', OR the other way around.

Please list the version of source used, and what exact command line used.

NOTE, if you re-run, do not do so until I get the next fix in, since it again will change things (remove all chances of that uninit value in split).

jfoug commented 9 years ago

I think other changes before doing cd15ea0 would have made this problem go away, but cd15ea0 was done just in case.

jfoug commented 9 years ago

I just did this command, and these were the only problems seen (cygwin64, OMP, without opencl)

$ ./jtrts.pl full -dynamic=all -q
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper 1.8.0.6-jumbo-1-565-gc0560a4+ OMP [cygwin 64-bit AVX-ac]
--------------------------------------------------------------------------------
form=mssql-cp850                  guesses: 1471 -show=1471 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=mssql-cp858                  guesses: 1471 -show=1471 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=mssql-cp437                  guesses: 1185 -show=1185 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=mssql-cp1252                 guesses: 1472 -show=1472 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
Some tests had Errors. Performed 751 tests.  4 errors
Time used was 4603 seconds
magnumripper commented 9 years ago

TS of c7b8fd034

$ ./jtrts.pl -dynamic=all -q full -pass=-dev=2
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper 1.8.0.6-jumbo-1-573-g4956a22 OMP [darwin14.4.0 64-bit AVX-ac]
--------------------------------------------------------------------------------
form=mssql-cp850                  guesses: 1471 -show=1471 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=mssql-cp858                  guesses: 1471 -show=1471 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=mssql-cp437                  guesses: 1185 -show=1185 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=mssql-cp1252                 guesses: 1472 -show=1472 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=sha256crypt-cuda             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
Self test failed (cmp_all(7))
.pot CHK:sha256crypt-cuda         guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=sha256crypt-cuda-new         guesses:    0 -show=   0 unk unk : Expected count(s) (1030)(1070)  [!!!FAILED2!!! exited, return code 1]
Self test failed (cmp_all(7))
.pot CHK:sha256crypt-cuda-new     guesses:    0 -show=   0 unk unk : Expected count(s) (1030)(1070)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=sha256crypt-cuda-xsalt       guesses:    0 -show=   0 unk unk : Expected count(s) (1264)(1296)  [!!!FAILED2!!! exited, return code 1]
Self test failed (cmp_all(7))
.pot CHK:sha256crypt-cuda-xsalt   guesses:    0 -show=   0 unk unk : Expected count(s) (1264)(1296)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=raw-md4-opencl               guesses:    0 -show= 172 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:raw-md4-opencl           guesses:    0 -show=  64 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (64 val-pwd  1 inval-pwd)
form=raw-md5-opencl               guesses:    0 -show= 172 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:raw-md5-opencl           guesses:    0 -show=  64 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (64 val-pwd  1 inval-pwd)
form=raw-sha1-opencl              guesses:    0 -show= 172 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:raw-sha1-opencl          guesses:    0 -show=  64 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (64 val-pwd  1 inval-pwd)
form=nt-opencl                    guesses:    0 -show= 171 unk unk : Expected count(s) (1478)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:nt-opencl                guesses:    0 -show=  64 unk unk : Expected count(s) (1478)  [!!!FAILED4!!!]  (64 val-pwd  1 inval-pwd)
form=mscash-opencl-utf8           guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:mscash-opencl-utf8       guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=sha256crypt-opencl-xsalt     guesses: 1290 -show=1290 0:00:00:35 DONE : Expected count(s) (1296)  [!!!FAILED1!!!]
.pot CHK:sha256crypt-opencl-xsalt guesses: 1288 -show=1288 0:00:00:16 DONE : Expected count(s) (1296)  [!!!FAILED4!!!]  (1288 val-pwd  0 inval-pwd)
form=pbkdf2-hmac-sha512-opencl    guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
Self test failed (cmp_all(1))
.pot CHK:pbkdf2-hmac-sha512-openc guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=odf-opencl                   guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
Self test failed (cmp_all(1))
.pot CHK:odf-opencl               guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=odf-opencl_1                 guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
Self test failed (cmp_all(1))
.pot CHK:odf-opencl_1             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=zip-opencl-type1             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
Self test failed (cmp_all(1))
.pot CHK:zip-opencl-type1         guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=zip-opencl-type2             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
Self test failed (cmp_all(1))
.pot CHK:zip-opencl-type2         guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=zip-opencl-type3             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
Self test failed (cmp_all(1))
.pot CHK:zip-opencl-type3         guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=zip-opencl                   guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
Self test failed (cmp_all(1))
.pot CHK:zip-opencl               guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=keychain-opencl              guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
Self test failed (cmp_all(1))
.pot CHK:keychain-opencl          guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=strip-opencl                 guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
Self test failed (cmp_all(1))
.pot CHK:strip-opencl             guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=Bcrypt-opencl                guesses:  300 -show= 300 0:00:00:01 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=Bcrypt_broken-opencl         guesses:  300 -show= 300 0:00:00:01 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
Some tests had Errors. Performed 821 tests.  24 errors  18 errors reprocessing the .POT files  17 runs had non-clean exit
Time used was 7971 seconds

The above is HD4000 for OpenCL. Re-run with nvidia:

$ ./jtrts.pl -q opencl -pass=-dev=2
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper 1.8.0.6-jumbo-1-573-g4956a22 OMP [darwin14.4.0 64-bit AVX-ac]
--------------------------------------------------------------------------------
form=raw-md4-opencl               guesses:    0 -show= 172 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:raw-md4-opencl           guesses:    0 -show=  64 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (64 val-pwd  1 inval-pwd)
form=raw-md5-opencl               guesses:    0 -show= 172 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:raw-md5-opencl           guesses:    0 -show=  64 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (64 val-pwd  1 inval-pwd)
form=raw-sha1-opencl              guesses:    0 -show= 172 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:raw-sha1-opencl          guesses:    0 -show=  64 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (64 val-pwd  1 inval-pwd)
form=nt-opencl                    guesses:    0 -show= 171 unk unk : Expected count(s) (1478)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:nt-opencl                guesses:    0 -show=  64 unk unk : Expected count(s) (1478)  [!!!FAILED4!!!]  (64 val-pwd  1 inval-pwd)
form=mscash-opencl-utf8           guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED2!!! exited, return code 1]
.pot CHK:mscash-opencl-utf8       guesses:    0 -show=   0 unk unk : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
Invalid negative number (-1) in chr at ./jtrts.pl line 732.DONE  [PASSED]               
Invalid negative number (-1) in chr at ./jtrts.pl line 732.
.pot CHK:netntlmv2-opencl-utf8    guesses: 1442 -show=1442 0:00:00:08 DONE : Expected count(s) (1442)(1500)  [!!!FAILED4!!!]  (1440 val-pwd  2 inval-pwd)
form=Bcrypt-opencl                guesses:  300 -show= 300 0:00:00:04 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=Bcrypt_broken-opencl         guesses:  300 -show= 300 0:00:00:04 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
Some tests had Errors. Performed 53 tests.  7 errors  6 errors reprocessing the .POT files  5 runs had non-clean exit
Time used was 4860 seconds

I wonder what is causing those "Invalid negative number" warnings.

frank-dittrich commented 9 years ago

On a fedora 22 system:

./john --list=build-info
Version: 1.8.0.6-jumbo-1-1274-ge14451e
Build: linux-gnu 64-bit AVX-ac OMP
SIMD: AVX, interleaving: MD4:3 MD5:3 SHA1:1 SHA256:1 SHA512:1
$ ./jtrts.pl -q
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.13, Dec 21, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper 1.8.0.6-jumbo-1-1274-ge14451e OMP [linux-gnu 64-bit AVX-ac]
--------------------------------------------------------------------------------
form=dynamic_20                   guesses: 1189 -show=1189 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=asa-md5                      guesses: 1189 -show=1189 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=formspring                   guesses:    0 -show=   0 N/A N/A : Expected count(s) (1500)  [!!!FAILED1!!!]
.pot CHK:formspring               guesses:    0 -show=   0 N/A N/A : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
form=episerver_sha1               guesses: 1476 -show=1476 0:00:00:03 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=episerver_sha1-utf8          guesses: 1403 -show=1403 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
form=episerver_sha256-utf8        guesses: 1437 -show=1437 0:00:00:00 DONE : Expected count(s) (1500)(1202)  [!!!FAILED1!!!]
form=ripemd-128                   guesses:    0 -show=   0 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED1!!!]
.pot CHK:ripemd-128               guesses:    0 -show=   0 N/A N/A : Expected count(s) (1500)  [!!!FAILED4!!!]  (0 val-pwd  1 inval-pwd)
Some tests had Errors. Performed 601 tests.  7 errors  2 errors reprocessing the .POT files
Time used was 2914 seconds
magnumripper commented 9 years ago

I think we should close this issue and open new ones. There should really not be any TS vs. Unicode problems in there now.

magnumripper commented 9 years ago

As far as I can see, the remaining issues are real issues and not TS problems.