NTLM bug?

[defuse]

On crackstation.net, try to crack...

0cb6948805f797bf2a82807973b89537 0e8231621f574d3636255ff36dd86c9c

The first one gives yellow and blank output (should be test), second one is correctly cracked as test2. Maybe it just happens to collide?

[defuse]

The hash of rest is also broken:

40cf2818bfbc0d49a1007cc3ede4728a

[defuse]

It's not all four-character strings because resz works: b8fc9f689efb579bfa6262a2df5351e6

[defuse]

By the way, crackstation.net doesn't actually use this repository's lookup code, but rather some hacked together thing I wrote way before trying to clean it up in this repository.

[defuse]

Ah, what's happening is it's getting a partial match for a string that's test<the byte 0xad>eksemplene which apparently somehow gets turned into the empty string when it gets displayed on crackstation.net. It's the same byte 0xad for both hashes that don't work.

[defuse]

I think it was an error in the creation of ntlm.idx: If you look at offset 841747606 in ntlm.idx you see the hash of "test" pointing to offset 12236501718 into realuniq.lst. At that offset is "test\xADeksemplene\x0Atest...". When the index was created it should have read that entire value and taken the NTLM hash of it, giving 31d6cfe0d16ae931b73c59d7e0c089c0 according to:

<?php

function ntlm($input, $raw)
{
    // Convert the password from UTF8 to UTF16 (little endian)
    $input=@iconv('UTF-8','UTF-16LE',$input);
    $MD4Hash=hash('md4',$input, $raw);
    return $MD4Hash;
}

$test = ntlm("test", false);
$testbad = ntlm("test\xADeksemplene", false);

var_dump($test);
var_dump($testbad);

(And indeed that's the value the script re-computes itself when it's determining whether it's a partial or impartial match)

[defuse]

So, I guess that '"\xAD" byte screwed up the iconv() or whatever, on whichever system generated the current ntlm.idx, and the index is wrong!

[defuse]

It'll probably get fixed on its own as I'm regenerating ntlm.idx on the server (so that I no longer have to host it at home) right now.

[defuse]

If I would have bothered to read the documentation for iconv, I would have seen that it returns false when there is an invalid character in the input string, which is what's happening here. False is then treated like the empty string somewhere else.

The root cause of this bug was me carelessly adding @ to the iconv to get around the error it was throwing. I found a link to the source code I copy/pasted by looking back through the commit history, and the URL is http://www.php.net/manual/en/ref.hash.php#82018. That comment is still there, and the code inside doesn't have this bug (and I checked with WayBackMachine that it hadn't been edited with a fix), so it was probably me who introduced the @.

It's fixed in #11.