Optimize handling of prepended salts of at least the fast hash's block size #saltfail

[magnumripper]

Nice post by Atom http://hashcat.net/forum/thread-4429.html

The context is ColdFusion's sha256(salt.sha1(pass)) but the hash functions are not important, it applies to things like MD5(salt.pass) as well. The flaw is it's a salt with a length that "equals or is greater than the blocksize of the hash" and that it's prepended to the password, as opposed to appended. This means, in JtR context, we can run the first digest operation in get_salt() and get it completely out of the hot loops.

We have formats that already use this optimization but we probably have some that don't. We should find them, list them here and fix them.

[jfoug]

This is the same optimization I have done for the PBKDF2-HMAC-* hashes. So any salts >= 64 bytes (or >= 128 bytes for 64 bit hashes), is a candidate for this, as long as salt is first. Even if salt is > 64 (say 80 bytes), it works. In that case, you store the prelim hash value, and the last 16 bytes of the salt.

I am pretty sure that JtR's salt dupe logic will also work, as long as the data is stored inline within the salt structure. in the case of 80 byte salt sha256, you would have 32 bytes of intermediate hash, and 16 bytes of the end of the salt.

Now, for SIMD formats, we may want to look at changing our SIMD functions (adding a flag), so that they would all reload from the same single input buffer.

[magnumripper]

Now, for SIMD formats, we may want to look at changing our SIMD functions (adding a flag), so that they would all reload from the same single input buffer.

That would probably be a very good feature. I recall having to copy the same data to all SIMD buffers in many formats (eg. hmac).

[jfoug]

I agree, it would be a good for this first limb is the same for all simd entries type hashes.

[magnumripper]

For iterated formats like PBKDF2 though, this would introduce another branch within hot code. But the new functionality would be used outside of the hot loop. Maybe it would end up a loss?

[magnumripper]

BTW why is salt size listed as zero in our Coldfusion format?

$ ../run/john --list=format-all-details -format:dynamic_1588
Format label                         dynamic_1588
 Disabled in configuration file      no
Min. password length in bytes        0
Max. password length in bytes        110
Min. keys per crypt                  1
Max. keys per crypt                  1680
Flags
 Case sensitive                      yes
 Supports 8-bit characters           yes
 Converts 8859-1 to UTF-16/UCS-2     no
 Honours --encoding=NAME             no
 False positives possible            no
 Uses a bitslice implementation      no
 The split() method unifies case     yes
 A $dynamic$ format                  yes (Flat buffer SIMD)
 A dynamic sized salt                no
 Parallelized with OpenMP            no
Number of test vectors               24
Algorithm name                       sha256($salt.sha1($pass)) (ColdFusion 11) 128/128 AVX 4x
Format name                          
Benchmark comment                    
Benchmark length                     0
Binary size                          16
Salt size                            0
Tunable cost parameters              
Example ciphertext                   $dynamic_1588$37F816D599BFD69C5A0D750198AB6E46E26CEB120C9AF3B1E5306515058CBAE8$D7B6D57262290BC0A634D2D1A0DFE59F1FBE47885DBC9BB1CEBA8EA9D09D9839

[jfoug]

Because there is no SaltLen value. I should look at this just to make sure. I added SaltLen=-128 and now I see this:

$ ../run/john --list=format-all-details -format:dynamic_1588
Format label                         dynamic_1588
 Disabled in configuration file      no
Min. password length in bytes        0
Max. password length in bytes        32
Min. keys per crypt                  1
Max. keys per crypt                  1680
Flags
 Case sensitive                      yes
 Supports 8-bit characters           yes
 Converts 8859-1 to UTF-16/UCS-2     no
 Honours --encoding=NAME             no
 False positives possible            no
 Uses a bitslice implementation      no
 The split() method unifies case     yes
 A $dynamic$ format                  yes (Flat buffer SIMD)
 A dynamic sized salt                no
 Parallelized with OpenMP            no
Number of test vectors               24
Algorithm name                       sha256($salt.sha1($pass)) (ColdFusion 11) 128/128 AVX 4x
Format name
Benchmark comment
Benchmark length                     0
Binary size                          16
Salt size                            128
Tunable cost parameters
Example ciphertext                   $dynamic_1588$37F816D599BFD69C5A0D750198AB6E46E26CEB120C9AF3B1E5306515058CBAE8$D7B6D57262290BC0A634D2D1A0DFE59F1FBE47885DBC9BB1CEBA8EA9D09D9839

[jfoug]

Is the salt len 'always' 64 bytes long? If so, then I could simply change it to SaltLen=64

[magnumripper]

According to Atom's post, it is.

[jfoug]

Found another candidate. dynamic_1503. NOTE, I created 2 new flags: MGF_KEYS_BASE16_IN1_SHA1 and MGF_KEYS_BASE16_IN1_SHA256 to handle these formats. b083104

But for 1503, we have reduced 3 limbs to 2 limbs. We can reduce that to 1 sha256 limb (in many salts mode), if we implement this 'salt-fail' optimization.

This is dyna_1503: sha256(sha256($pass).$salt) (XenForo SHA-256). This one is 'slightly' different. It is more like pbkdf2-hmac type functions. But we certainly can reduce the hash limbs for multi-salts.

[magnumripper]

That's another kind of optimization, not #saltfail. But whatever we call it, it's good of course.