search

openwall / john

John the Ripper jumbo - advanced offline password cracker, which supports hundreds of hash and cipher types, and runs on many operating systems, CPUs, GPUs, and even some FPGAs
https://www.openwall.com/john/
Other
9.98k stars 2.06k forks source link

-show is not always cannonicalizing values #751

Closed jfoug closed 9 years ago

jfoug commented 9 years ago

Found in TS when I added code to do -show | tail 1 to find the count that way. There are several formats that have more shows. 

$ ./jtrts.pl -no-prelims -q
-------------------------------------------------------------------------------
- JtR-TestSuite (jtrts). Version 1.12.14, June 30, 2014.  By, Jim Fougeron & others
- Testing:  John the Ripper password cracker, version 1.8.0.2-bleeding-jumbo [cygwin 64-bit SSSE3-autoconf]
--------------------------------------------------------------------------------
form=dynamic_0                    guesses: 1500 -show=1502 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED!!!]
form=dynamic_71                   guesses: 1500 -show=1502 0:00:00:00 DONE : Expected count(s) (1500)  [!!!FAILED!!!]

... I will fill in the others when I run full test suite

In the dyna_0 case, there is one hash replicated 3 times, with case changes to the hash value.

u18-dynamic_0:$dynamic_0$cb803b67f0eb50ad7a88d4d320633e1a:18:0:dffffffffffff:: u18-dynamic_0:$dynamic_0$CB803b67f0eb50ad7a88d4d320633e1a:18:0:dffffffffffff:: u18-dynamic_0:$dynamic_0$cB803b67f0eb50ad7a88d4d320633e1a:18:0:dffffffffffff::

Dyna_71 was same. I had some case change in one of the hashes.

the crack cannonicalizes these down to 1 hash, but -show is not.

frank-dittrich commented 9 years ago

Dynamic formats that use hex encoding should unify case in split() and set FMT_SPLIT_UNIFIES_CASE.

jfoug commented 9 years ago

I made some simply changes to dyna's dynamic_SETUP() function (around line 6910 in dynamic_fmt.c) to output what split is being used. Here is a quick run.

I have to make SURE that split_UC is working properly. But if so, then dyna IS working as it should (I think).

dynamic_0: md5($p) (raw-md5)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1: md5($p.$s) (joomla)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2: md5(md5($p)) (e107)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_3: md5(md5(md5($p)))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_4: md5($s.$p) (OSC)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_5: md5($s.$p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_6: md5(md5($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_8: md5(md5($s).$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_9: md5($s.md5($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_10: md5($s.md5($s.$p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_11: md5($s.md5($p.$s))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_12: md5(md5($s).md5($p)) (IPB)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_13: md5(md5($p).md5($s))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_14: md5($s.md5($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_15: md5($u.md5($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_16: md5(md5(md5($p).$s).$s2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_17: phpass ($P$ or $H$)  split set to split()
dynamic_18: md5($s.Y.$p.0xF7.$s) (Post.Office MD5)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_19: Cisco PIX (MD5)  split set to split()
dynamic_20: Cisco ASA (MD5 salted)  split set to split()
dynamic_21: HTTP Digest Access Auth  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_22: md5(sha1($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_23: sha1(md5($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_24: sha1($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_25: sha1($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_26: sha1($p) raw-sha1  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_29: md5(unicode($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_30: md4($p) (raw-md4)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_31: md4($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_32: md4($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_33: md4(unicode($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_34: md5(md4($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_35: sha1(uc($u).:.$p) (ManGOS)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_36: sha1($u.:.$p) (ManGOS2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_37: sha1(lc($u).$p) (SMF)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_38: sha1($s.sha1($s.sha1($p))) (Wolt3BB)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_39: md5($s.pad16($p)) (net-md5)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_40: sha1($s.pad16($p)) (net-sha1)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_50: sha224($p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_51: sha224($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_52: sha224($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_53: sha224(sha224($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_54: sha224(sha224_raw($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_55: sha224(sha224($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_56: sha224($s.sha224($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_57: sha224(sha224($s).sha224($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_58: sha224(sha224($p).sha224($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_60: sha256($p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_61: sha256($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_62: sha256($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_63: sha256(sha256($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_64: sha256(sha256_raw($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_65: sha256(sha256($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_66: sha256($s.sha256($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_67: sha256(sha256($s).sha256($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_68: sha256(sha256($p).sha256($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_70: sha384($p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_71: sha384($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_72: sha384($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_73: sha384(sha384($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_74: sha384(sha384_raw($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_75: sha384(sha384($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_76: sha384($s.sha384($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_77: sha384(sha384($s).sha384($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_78: sha384(sha384($p).sha384($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_80: sha512($p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_81: sha512($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_82: sha512($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_83: sha512(sha512($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_84: sha512(sha512_raw($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_85: sha512(sha512($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_86: sha512($s.sha512($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_87: sha512(sha512($s).sha512($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_88: sha512(sha512($p).sha512($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_90: GOST($p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_91: GOST($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_92: GOST($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_93: GOST(GOST($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_94: GOST(GOST_raw($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_95: GOST(GOST($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_96: GOST($s.GOST($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_97: GOST(GOST($s).GOST($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_98: GOST(GOST($p).GOST($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_100: WHIRLPOOL($p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_101: WHIRLPOOL($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_102: WHIRLPOOL($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_103: WHIRLPOOL(WHIRLPOOL($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_104: WHIRLPOOL(WHIRLPOOL_raw($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_105: WHIRLPOOL(WHIRLPOOL($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_106: WHIRLPOOL($s.WHIRLPOOL($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_107: WHIRLPOOL(WHIRLPOOL($s).WHIRLPOOL($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_108: WHIRLPOOL(WHIRLPOOL($p).WHIRLPOOL($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_110: Tiger($p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_111: Tiger($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_112: Tiger($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_113: Tiger(Tiger($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_114: Tiger(Tiger_raw($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_115: Tiger(Tiger($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_116: Tiger($s.Tiger($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_117: Tiger(Tiger($s).Tiger($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_118: Tiger(Tiger($p).Tiger($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_120: RIPEMD128($p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_121: RIPEMD128($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_122: RIPEMD128($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_123: RIPEMD128(RIPEMD128($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_124: RIPEMD128(RIPEMD128_raw($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_125: RIPEMD128(RIPEMD128($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_126: RIPEMD128($s.RIPEMD128($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_127: RIPEMD128(RIPEMD128($s).RIPEMD128($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_128: RIPEMD128(RIPEMD128($p).RIPEMD128($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_130: RIPEMD160($p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_131: RIPEMD160($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_132: RIPEMD160($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_133: RIPEMD160(RIPEMD160($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_134: RIPEMD160(RIPEMD160_raw($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_135: RIPEMD160(RIPEMD160($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_136: RIPEMD160($s.RIPEMD160($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_137: RIPEMD160(RIPEMD160($s).RIPEMD160($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_138: RIPEMD160(RIPEMD160($p).RIPEMD160($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_140: RIPEMD256($p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_141: RIPEMD256($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_142: RIPEMD256($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_143: RIPEMD256(RIPEMD256($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_144: RIPEMD256(RIPEMD256_raw($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_145: RIPEMD256(RIPEMD256($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_146: RIPEMD256($s.RIPEMD256($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_147: RIPEMD256(RIPEMD256($s).RIPEMD256($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_148: RIPEMD256(RIPEMD256($p).RIPEMD256($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_150: RIPEMD320($p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_151: RIPEMD320($s.$p)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_152: RIPEMD320($p.$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_153: RIPEMD320(RIPEMD320($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_154: RIPEMD320(RIPEMD320_raw($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_155: RIPEMD320(RIPEMD320($p).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_156: RIPEMD320($s.RIPEMD320($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_157: RIPEMD320(RIPEMD320($s).RIPEMD320($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_158: RIPEMD320(RIPEMD320($p).RIPEMD320($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1001 md5(md5(md5(md5($p))))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1002 md5(md5(md5(md5(md5($p)))))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1003 md5(md5($p).md5($p))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1004 md5(md5(md5(md5(md5(md5($p))))))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1005 md5(md5(md5(md5(md5(md5(md5($p)))))))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1006 md5(md5(md5(md5(md5(md5(md5(md5($p))))))))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1007 md5(md5($p).$s) (vBulletin)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1008 md5($p.$s) (RADIUS User-Password)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1009 md5($s.$p) (RADIUS Responses)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1010 md5($p null_padded_to_len_100) RAdmin v2.x MD5  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1011 md5($p.md5($s)) (WebEdition CMS)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1012 md5($p.md5($s)) (WebEdition CMS)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1013 md5($p.PMD5(username)) (WebEdition CMS)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1014 md5($p.$s) (long salt)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1015 md5(md5($p.$u).$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1018 md5(sha1(sha1($pass)))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1019 md5(sha1(sha1(md5($pass))))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1020 md5(sha1(md5($pass)))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1021 md5(sha1(md5(sha1($pass))))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1022 md5(sha1(md5(sha1(md5($pass)))))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1023 sha1($pass) (first 32 bytes)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1024 sha1(md5($pass)) (first 32 bytes)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1025 sha1(md5(md5($pass))) (first 32 bytes)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1026 sha1(sha1($pass))) (first 32 bytes)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1027 sha1(sha1(sha1($pass)))) (first 32 bytes)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1028 sha1(sha1_raw($pass)) (first 32 bytes)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1029 sha256($pass) (first 32 bytes)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1030 Whirlpool($pass) (first 32 bytes)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1031 GOST($pass) (first 32 bytes)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1300 md5(md5_raw($pass))  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1350 md5(md5($s.$p):$s)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1400 sha1(unicode($p)) (Microsoft CREDHIST)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1401 md5($u.\nskyper\n.$p) (Skype MD5)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1501 sha1($salt.sha1($pass) (Redmine)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1502 sha1(sha1($pass).$salt) (XenForo SHA-1)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_1503 sha256(sha256($pass).$salt) (XenForo SHA-256)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2000 md5($p) (PW > 55 bytes, sse2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2001 md5($p.$s) (joomla) (PW > 23 bytes, sse2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2002 md5(md5($p)) (e107) (PW > 55 bytes, sse2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2003 md5(md5(md5($p))) (PW > 55 bytes, sse2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2004 md5($s.$p) (OSC) (PW > 31 bytes, sse2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2005 md5($s.$p.$s) (PW > 31 bytes, sse2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2006 md5(md5($p).$s) (PW > 55 bytes, sse2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2008 md5(md5($s).$p) (PW > 23 bytes, sse2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2009 md5($s.md5($p)) (PW > 55 or salt > 23 bytes, sse2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2010 md5($s.md5($s.$p)) (PW > 32 or salt > 23 bytes, sse2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2011 md5($s.md5($p.$s)) (PW > 32 or salt > 23 bytes, sse2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
dynamic_2014 md5($s.md5($p).$s) (PW > 55 or salt > 11 bytes, sse2)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()

I also ran, where I listed a single dyna format (just in case if a specific was asked for, this linkage code was not working).

$ ../run/john -test -form=dynamic_0
dynamic_0: md5($p) (raw-md5)  Setting FMT_SPLIT_UNIFIES_CASE  split set to split_UC()
Benchmarking: dynamic_0 [md5($p) (raw-md5) 128/128 SSSE3 10x4x3]... DONE

Worked just fine. Now if split_UC is right (I am pretty sure it is), then this is a bug in john, not in dyna.

jfoug commented 9 years ago

I think this is the same 'bug' (IMHO), which we get when we do not use the -w.

here is a simple test:

cat in

u18-dynamic_0:$dynamic_0$cb803b67f0eb50ad7a88d4d320633e1a:18:0:dffffffffffff:: u18-dynamic_0:$dynamic_0$CB803b67f0eb50ad7a88d4d320633e1a:18:0:dffffffffffff:: u18-dynamic_0:$dynamic_0$cB803b67f0eb50ad7a88d4d320633e1a:18:0:dffffffffffff::

$ ../run/john in Warning: detected hash type "dynamic_0", but the string is also recognized as "Raw-MD5" Use the "--format=Raw-MD5" option to force loading these as that type instead Loaded 3 password hashes with no different salts (dynamic_0 [md5($p) (raw-md5) 128/128 SSSE3 10x4x3]) Press 'q' or Ctrl-C to abort, almost any other key for status dffffffffffff (u18-dynamic_0) dffffffffffff (u18-dynamic_0) dffffffffffff (u18-dynamic_0) 3g 0:00:00:00 DONE 1/3 (2014-09-21 10:45) 375.0g/s 2000p/s 2000c/s 6000C/s u0..ddynamic Use the "--show" option to display all of the cracked passwords reliably

$ rm ../run/john.pot

$ ../run/john in -w=../test/pw.dic Warning: detected hash type "dynamic_0", but the string is also recognized as "Raw-MD5" Use the "--format=Raw-MD5" option to force loading these as that type instead Loaded 1 password hash (dynamic_0 [md5($p) (raw-md5) 128/128 SSSE3 10x4x3]) Press 'q' or Ctrl-C to abort, almost any other key for status dffffffffffff (u18-dynamic_0) 1g 0:00:00:00 DONE (2014-09-21 10:45) 166.6g/s 20000p/s 20000c/s 20000C/s Skippin▒ an▒*..a123456789012345678901234567890123456789012345678901234 Warning: passwords printed above might not be all those cracked Use the "--show" option to display all of the cracked passwords reliably Session completed

$ ../run/john in -show u18-dynamic_0:dffffffffffff:18:0:dffffffffffff:: u18-dynamic_0:dffffffffffff:18:0:dffffffffffff:: u18-dynamic_0:dffffffffffff:18:0:dffffffffffff::

3 password hashes cracked, 0 left

Dyna is working properly. -show is not. Also, IMHO, 'single' mode JtR is also not working right, but magnum has listed reasons why dupe removal is not done there, and I can live with it.

But I think -show is failing to have dupe logic upon the input file load.

frank-dittrich commented 9 years ago

@jfoug: sorry, I was wrong. Dynamic was doing everything as it should. That's why the pot file got 1500 lines. But --show is reporting how many users got their passwords cracked. So, evben with exactly the same salt, john --show would report more cracked passwords than lines in the pot file.

jfoug commented 9 years ago

So this behavior is also by design?

frank-dittrich commented 9 years ago

Yes, I am absolutely sure it is.

jfoug commented 9 years ago

Then I will PEBCAK and close it.

magnumripper commented 9 years ago

As discussed before, Single mode (so batch mode too) will show a number of loaded hashes without salt/hash dupe suppression. This added to your confusion again (and has confused me on countless occasions).

$ ../run/john in
Warning: detected hash type "dynamic_0", but the string is also recognized as "Raw-MD5"
Use the "--format=Raw-MD5" option to force loading these as that type instead
Loaded 3 password hashes with no different salts (dynamic_0 [md5($p) (raw-md5) 128/128 AVX 10x4x3])
...
$ ../run/john in -w
Warning: detected hash type "dynamic_0", but the string is also recognized as "Raw-MD5"
Use the "--format=Raw-MD5" option to force loading these as that type instead
Loaded 1 password hash (dynamic_0 [md5($p) (raw-md5) 128/128 AVX 10x4x3])
...