search

openwall / john

John the Ripper jumbo - advanced offline password cracker, which supports hundreds of hash and cipher types, and runs on many operating systems, CPUs, GPUs, and even some FPGAs
https://www.openwall.com/john/
Other
10.36k stars 2.11k forks source link

pkzip non-handled compression type #2528

Closed kestufabrix closed 7 years ago

kestufabrix commented 7 years ago

Hi, I need to crack a zip archive password containing two files, a small .txt and another big .zip. The password seems to be zipcrypto

I used zip2john to get the hash of the archive but I get : ver 1.0 XXX.zip->YYY.txt PKZIP Encr: cmplen=51, decmplen=39, crc=BDF11DE0 ver 4.5 ~/Downloads/00-Utilitaires/XXX.zip->ZZZ.zip is not encrypted, or stored with non-handled compression type (though it does ask for a password when unzipping)

the hash I get is :

XXX.zip:$pkzip2$1*1*2*0*33*27*bdf11de0*0*2c*0*33*bdf1*aa8a*e2f57a8f225245b7aa7ec500c0397f9a412f501c3f8ea362aa5dbf7fc0871054c2e305b60cf04c10575b6837737db7c5fbb609*$/pkzip2$:::::XXX.zip

John found a password within 3 hours but it won't work... Would you have any idea where the problem might be? (I must admit I can't be sure my archive is not broken...)

Thank you so much in advance...

Note I use an up-to-date bleeding jumbo version of john (though the "$john --list=build-info" command returns "unknown commands") on Ubuntu 17.04.

magnumripper commented 7 years ago

Any chance you can share that archive with us (privately)?

kestufabrix commented 7 years ago

Yes of course. Though it's a big file and its origin is not really secure nor very official... Honestly my purpose is just testing but it's certainly still not very very lawful... Let me know how to share it (dropbox?).

magnumripper commented 7 years ago

In that case I think I'll pass. Another idea though: It looks to me from your output that the archive may contain an encrypted file "YYY.txt" and an unencrypted "ZZZ.zip". What if you try to extract only the latter one?

kestufabrix commented 7 years ago

Thanks for your answer. Unfortunately the ZZZ.zip does asks for a password! That's what troubles me...

claudioandre-br commented 7 years ago

JtR and the 2john can handle the not encrypted, or stored with non-handled compression type. I just tried to reproduce your use case. 

ver 1.0 efh 5455 efh 7875 XXX.zip->YYY.txt PKZIP Encr: 2b chk, TS_chk, cmplen=16, decmplen=4, crc=7F808508
ver 4.5 XXX.zip->ZZZ.zip is not encrypted, or stored with non-handled compression type
XXX.zip:$pkzip2$1*2*2*0*10*4*7f808508*0*41*0*10*7f80*7319*8fe32c524ec41bc23f18d30fd3641020*$/pkzip2$:::::XXX.zip

So, it has to be something else.

BTW: are your saying that you can crack the hash you posted using something like this ./john ~/hash -mask=zipcrypto?

claudioandre-br commented 7 years ago

Anyway, I was able to do something really interesting (the same salt for 2 different passwords):

XXX.zip:$pkzip2$1*2*2*0*10*4*7f808508*0*41*0*10*7f80*7319*8fe32c524ec41bc23f18d30fd3641020*$/pkzip2$:::::XXX.zip
XXX.zip:$pkzip2$1*2*2*0*10*4*7f808508*0*41*0*10*7f80*7319*30fc1e61e10e8d79ec55a90d58121392*$/pkzip2$:::::/home/claudio/Downloads/XXX.zip

For this hash file, JtR finds the wrong password for the 2nd line and can't find the right one.

magnumripper commented 7 years ago

@claudioandre that's weird, we need to look into that. When you say "same salt" you mean same actual pkzip salt but in JtR's perspective it's not regarded as a salt in normal sense, the whole hash is (and binary size is 0 -- what we use to call a non-hash).

@kestufabrix could you post the output of zipinfo -v XXX.zip? It should tell all details about encryption types, versions and so on.

kestufabrix commented 7 years ago

Actually john did crack the hash but without the zipcrypto mask. But anyway the password it finds won't work.

The output of zipinfo is:

Archive:  XXX.zip
There is no zipfile comment.

End-of-central-directory record:
-------------------------------

  Zip archive file size:                2831753269 (00000000A8C92035h)
  Actual end-cent-dir record offset:    2831753171 (00000000A8C91FD3h)
  Expected end-cent-dir record offset:  2831753171 (00000000A8C91FD3h)
  (based on the length of the central directory and its expected offset)

  This zipfile constitutes the sole disk of a single-part archive; its
  central directory contains 2 entries.
  The central directory is 252 (00000000000000FCh) bytes long,
  and its (expected) offset in bytes from the beginning of the zipfile
  is 2831752919 (00000000A8C91ED7h).

Central directory entry #1:
---------------------------

  YYY.txt

  offset of local header from start of archive:   0
                                                  (0000000000000000h) bytes
  file system or operating system of origin:      MS-DOS, OS/2 or NT FAT
  version of encoding software:                   3.1
  minimum file system compatibility required:     MS-DOS, OS/2 or NT FAT
  minimum software version required to extract:   1.0
  compression method:                             none (stored)
  file security status:                           encrypted
  extended local header:                          yes                                                                                                                    
  file last modified on (DOS date/time):          2016 Oct 14 21:20:20                                                                                                   
  32-bit CRC value (hex):                         bdf11de0                                                                                                               
  compressed size:                                51 bytes                                                                                                               
  uncompressed size:                              39 bytes                                                                                                               
  length of filename:                             14 characters                                                                                                          
  length of extra field:                          36 bytes                                                                                                               
  length of file comment:                         0 characters
  disk number on which file begins:               disk 1
  apparent file type:                             binary
  non-MSDOS external file attributes:             000000 hex
  MS-DOS file attributes (20 hex):                arc 

  The central-directory extra field contains:
  - A subfield with ID 0x000a (PKWARE Win32) and 32 data bytes.  The first
    20 are:   00 00 00 00 01 00 18 00 c9 79 a3 71 9b 26 d2 01 f4 d1 14 a4.

  There is no file comment.

Central directory entry #2:
---------------------------

  There are an extra -20 bytes preceding this file.

  ZZZ.zip

  offset of local header from start of archive:   111
                                                  (000000000000006Fh) bytes
  file system or operating system of origin:      MS-DOS, OS/2 or NT FAT
  version of encoding software:                   3.1
  minimum file system compatibility required:     MS-DOS, OS/2 or NT FAT
  minimum software version required to extract:   4.5
  compression method:                             none (stored)
  file security status:                           encrypted
  extended local header:                          yes
  file last modified on (DOS date/time):          2016 Oct 9 23:41:54
  32-bit CRC value (hex):                         86adc65e
  compressed size:                                2831752680 bytes
  uncompressed size:                              2831752668 bytes
  length of filename:                             54 characters
  length of extra field:                          56 bytes
  length of file comment:                         0 characters
  disk number on which file begins:               disk 1
  apparent file type:                             binary
  non-MSDOS external file attributes:             000000 hex
  MS-DOS file attributes (20 hex):                arc 

  The central-directory extra field contains:
  - A subfield with ID 0x0001 (PKWARE 64-bit sizes) and 16 data bytes:
    dc 1d c9 a8 00 00 00 00 e8 1d c9 a8 00 00 00 00.
  - A subfield with ID 0x000a (PKWARE Win32) and 32 data bytes.  The first
    20 are:   00 00 00 00 01 00 18 00 e8 dd 27 64 c1 22 d2 01 7f 88 bb 13.

  There is no file comment.
magnumripper commented 7 years ago

OK, here's a wild theory: Maybe our zip2john and/or the pkzip format have "32-bit issues". The size of ZZZ.zip can't be fit in a signed int.

kestufabrix commented 7 years ago

I admit I thought of that 32/64 bits issue but it is far beyond my skills and undestanding to solve the problem then! :-)

claudioandre-br commented 7 years ago

When you say "same salt" you mean

I mean JtR said that to me. 

Using default input encoding: UTF-8
Loaded 2 password hashes with no different salts (PKZIP [32/64])
Remaining 1 password hash
Will run 2 OpenMP threads
magnumripper commented 7 years ago

I mean JtR said that to me.

Wow that is a bug. I guess Jim is setting the dyna salt size too short.

I will have a look at both of these issues tomorrow. If it's a 32-bit problem it should be easy to fix.

claudioandre-br commented 7 years ago

There a some stuff to clarify:

magnumripper commented 7 years ago

Very good point. Another way to achieve the same is (or should be) zip2john -o YYY.txt XXX.zip to get the hash. Then crack it and try unzipping the big file with the same password.

magnumripper commented 7 years ago

No wait, the hash you got in the first place was just that. It's the hash of the small txt file.

claudioandre-br commented 7 years ago

So, if he creates a new zip, we can compare the 2 hashes and discard (or be sure about) some kind of bugs.

kestufabrix commented 7 years ago

OK then, I deleted the big ZZZ.zip file. The hash I get then from zip2john is :

XXX.zip:$pkzip2$1*1*2*0*33*27*bdf11de0*0*2c*0*33*bdf1*aa8a*e2f57a8f225245b7aa7ec500c0397f9a412f501c3f8ea362aa5dbf7fc0871054c2e305b60cf04c10575b6837737db7c5fbb609*$/pkzip2$:::::XXX.zip

And now john is cooking... I'll let you know.

magnumripper commented 7 years ago

That's the same hash

kestufabrix commented 7 years ago

Ah yes it's the same hash, sorry, I checked too fast...

magnumripper commented 7 years ago

Some conclusions so far:

solardiz commented 7 years ago

On Sat, May 06, 2017 at 03:52:59AM -0700, magnum wrote:

  • Both files are stored, not inflated. In that case our only validation of a correct password is the 32-bit CRC. This means we'll have a false positive for every 4 billion guesses (on average) and that's what you saw. Adding the --keep-guessing option to john will make it go on and find more alternatives. When you find a password that works with unrar for both files, it's the correct one (I'm assuming they are having the same one).

In cases like this - with multiple files in a zip - would it be possible for us to have multi-file "hashes" and filter out guesses based on multiple 32-bit CRCs at once?

magnumripper commented 7 years ago

would it be possible for us to have multi-file "hashes" and filter out guesses based on multiple 32-bit CRCs at once?

I believe that is actually what would happen with current code, if both files were supported. At least unless we disqualify the second file based on its huge size. I will look into that once I get support for v4.5 and 64-bit sizes.

kestufabrix commented 7 years ago

John is still running... 40 hours now. It did find a second password but still not the good one. Id does try to extract the .txt file though but the .txt file seems corrupted with wrong UTF-8 characters... And the password won't work with the ZZZ.zip file anyway (I still assume it's the same password).

magnumripper commented 7 years ago

@kestufabrix if you ever crack it, please send the password to me (off list if you wish) so I can verify some changes to the format.

magnumripper commented 7 years ago

@kestufabrix I have committed a change that supports the larger file. The result of that, like Solar mentioned, is merely that you will get less false positives (it will verify both files' CRC). I have tested this with some mockup files and it seems to work. It also did not break any old samples we had. You can keep running with what you've got, or you could try stopping the job, re-creating the input file (with updated zip2john) and then resume the job.

As always, no guarantees ;-)

magnumripper commented 7 years ago

I believe zip2john now fully supports 64-bit sizes. The pkzip format itself doesn't, in case any actual size exceeds 4 GB. This is a to-do and I'm working on it. We seem to have core code limitations (despite our "long line support").

magnumripper commented 7 years ago

After the above commits and finally efae666, all problems I know of are now fixed.