Closed kholia closed 7 years ago
The bug seems to be in Symmetrically_Encrypted_Data_Packet
function (affected function). It seems that inappropriate code from Symmetrically_Encrypted_and_MDC_Packet
function was inserted into the affected function.
diff --git a/src/gpg2john.c b/src/gpg2john.c
index efd8959..6517e80 100644
--- a/src/gpg2john.c
+++ b/src/gpg2john.c
@@ -934,11 +934,8 @@ Symmetrically_Encrypted_Data_Packet(int len, int first, int partial, char *hash)
if (first) {
cp = hash;
totlen = 0;
- // printf("\tVer %d\n", Getc());
- Getc(); // version (we only read this from the first packet. Not read from rest of the 'partial' packets.
- } else
- ++len; // we want the 'full' length for subsquent partial packets, since the logic is len-1 we simply fake it out.
- totlen += (len-1);
+ }
+ totlen += (len);
switch (mode) {
case SYM_ALG_MODE_NOT_SPECIFIED:
@@ -963,7 +960,7 @@ Symmetrically_Encrypted_Data_Packet(int len, int first, int partial, char *hash)
if (give(len, m_data, sizeof(m_data)) != len)
return;
- cp += print_hex(m_data, len - 1, cp);
+ cp += print_hex(m_data, len, cp);
if (!partial) {
// we only dump the packet out when we get the 'non-partial' packet (i.e. last one).
Here is a symmetrically encrypted GPG file which gets cracked only after applying this patch.
I am not sure if this patch solves the problem 100% but it is a start.
Tip: Use pgpdump
(gpg2john is based on it) to rule out parsing problems with the original files.
I applied the patch and we have some functionality. I was able to test your file as well as two of my own. Your file and one of mine work. One of mine does not, because it large at 3GB.
gpg2john is creating huge files. with a small 197MB file, it becomes a 393MB hash. When I try this on a large file of 3GB it becomes a 9GB hash which john will not accept.
$ ls -lh data2*
-rw-r--r-- 1 user staff 393M 7 Jul 20:18 data2.hash
-rw-r--r-- 1 user staff 197M 7 Jul 20:27 data2.tgz.gpg
Also, I didn't know exactly how I am supposed to use pgpdump. Is it supposed to be used on the hash? pgpdump example.hash
? because that just gives me pgpdump: can't find PGP armor boundary.
for interest here are some pgpdump info on my files.
3GB file:
$ pgpdump 3GB_file.tgz.gpg
Old: Symmetric-Key Encrypted Session Key Packet(tag 3)(13 bytes)
New version(4)
Sym alg - CAST5(sym 3)
Iterated and salted string-to-key(s2k 3):
Hash alg - SHA1(hash 2)
Salt - 07 3a 52 c3 0b a4 10 ea
Count - 65536(coded count 96)
New: Symmetrically Encrypted Data Packet(tag 9)(8192 bytes) partial start
Encrypted data [sym alg is specified in sym-key encrypted session key]
New: (8192 bytes) partial continue
New: (4096 bytes) partial continue
New: (2048 bytes) partial continue
New: (512 bytes) partial continue
New: (485 bytes) partial end
196MB file:
$ pgpdump data2.tgz.gpg
Old: Symmetric-Key Encrypted Session Key Packet(tag 3)(13 bytes)
New version(4)
Sym alg - CAST5(sym 3)
Iterated and salted string-to-key(s2k 3):
Hash alg - SHA1(hash 2)
Salt - d8 56 89 b0 ba f0 88 66
Count - 19922944(coded count 227)
New: Symmetrically Encrypted Data Packet(tag 9)(8192 bytes) partial start
Encrypted data [sym alg is specified in sym-key encrypted session key]
New: (8192 bytes) partial continue
New: (4096 bytes) partial continue
New: (2048 bytes) partial continue
New: (512 bytes) partial continue
New: (485 bytes) partial end
your example:
$ pgpdump example_1_1234.gpg
Old: Symmetric-Key Encrypted Session Key Packet(tag 3)(13 bytes)
New version(4)
Sym alg - CAST5(sym 3)
Iterated and salted string-to-key(s2k 3):
Hash alg - SHA1(hash 2)
Salt - 90 bd 75 05 24 9c 47 98
Count - 65536(coded count 96)
New: Symmetrically Encrypted Data Packet(tag 9)(8192 bytes) partial start
Encrypted data [sym alg is specified in sym-key encrypted session key]
New: (1024 bytes) partial continue
New: (512 bytes) partial continue
New: (301 bytes) partial end
Working cracked file example here:
$ unzip example_1_1234.gpg.zip
Archive: example_1_1234.gpg.zip
extracting: example_1_1234.gpg
$ ./gpg2john example_1_1234.gpg > example.hash
[gpg2john] MDC is misssing, expect false positives!
$ pgpdump example.hash
pgpdump: can't find PGP armor boundary.
$ ls -lh example*
-rw-r--r-- 1 user staff 20K 7 Jul 20:21 example.hash
-rw-rw-r-- 1 user staff 9.8K 7 Jul 15:00 example_1_1234.gpg
-rw-r--r--@ 1 user staff 10K 7 Jul 17:47 example_1_1234.gpg.zip
$ ./john example.hash
Warning: detected hash type "gpg", but the string is also recognized as "gpg-opencl"
Use the "--format=gpg-opencl" option to force loading these as that type instead
Using default input encoding: UTF-8
Loaded 1 password hash (gpg, OpenPGP / GnuPG Secret Key [32/64])
Will run 8 OpenMP threads
Press 'q' or Ctrl-C to abort, almost any other key for status
1234 (?)
1g 0:00:00:00 DONE 2/3 (2017-07-07 20:24) 25.00g/s 12800p/s 12800c/s 12800C/s 123456..crawford
Use the "--show" option to display all of the cracked passwords reliably
Session completed
Quick note: Run pgpdump
on original .gpg
files, and not on hashes. Our gpg2john
tool is a modified version of pgpdump. Running pgpdump won't extract hashes. I suggested running pgpdump to rule out basic parsing problems with our/your gpg files.
I don't think there is a reasonable way to handle / crack such large .gpg
files currently.
However, I might have a new idea to try here. Since, there is no MDC, we actually don't need to generate a full hash which includes all 3 GB of the original data, especially when we only use 4 bytes of it, currently! We could generate a truncated hash (i.e. limited to 64 KB), and process such hashes with JtR.
The bigger problem is that cracking of symmetrically encrypted files lacking MDC is currently not practically possible due to too many false positives involved. See #1892 for details. You will run into this problem eventually.
Maybe https://github.com/kholia/PGPCrack-NG can work but it has never been tested under such "extreme" conditions.
Let us know if you find a software solution which works better for you!
Thanks for the feedback. I will test PGPCrack-NG with a very large file and see if it has any problems.
Would prefer the possibility of using JtR however.
I tested with PGPCrack-NG on a 3GB gpg cast5 without MDC and it worked as expected. No errors.
Is there an open issue for me to track progress of this idea to add the functionality you mentioned?
"We could generate a truncated hash (i.e. limited to 64 KB), and process such hashes with JtR"
Glad to hear that PGPCrack-NG is working :-)
You can keep an eye on https://github.com/magnumripper/JohnTheRipper/issues/1892 but there has been no progress on it in a long time.
I tried to understand the inner workings of PGPCrack-NG last weekend but didn't make much progress. For now, I recommend using PGPCrack-NG.
I ran into this problem. In case it helps with the bug hunt:
$ run/gpg2john 1gig_of_data.tar.gpg > /tmp/1gig_of_data.tar.gpg.jtrhash
File 1gig_of_data.tar.gpg
[gpg2john] MDC is missing, expect lots of false positives!
$ run/john --wordlist=/tmp/wdlst /tmp/1gig_of_data.tar.gpg.jtrhash
Segmentation fault
$ pgpdump 1gig_of_data.tar.gpg | uniq
Old: Symmetric-Key Encrypted Session Key Packet(tag 3)(13 bytes)
New version(4)
Sym alg - CAST5(sym 3)
Iterated and salted string-to-key(s2k 3):
Hash alg - SHA1(hash 2)
Salt - cf ac 66 53 14 0e 72 ae
Count - 65536(coded count 96)
New: Symmetrically Encrypted Data Packet(tag 9)(8192 bytes) partial start
Encrypted data [sym alg is specified in sym-key encrypted session key]
New: (8192 bytes) partial continue
New: (4096 bytes) partial continue
New: (1024 bytes) partial continue
New: (512 bytes) partial continue
New: (191 bytes) partial end
The hash produced by gpg2john
was 4gb (four times bigger than the source content).
So I started to follow the pgpcrack-ng suggestion. There's a problem building PGPCrack-NG on debian. I took these steps:
aptitude install libpth-dev libbz2-dev libassuan-dev libgcrypt20-dev libgpg-error-dev
cd ~/src/
git clone https://github.com/kholia/PGPCrack-NG.git
/usr/include/libassuan2
does not exist, so updated Makefile
to reference the directory for libassuan.a
and libassuan.so(
/usr/lib/x86_64-linux-gnu/`).make
gcc -I/usr/lib/x86_64-linux-gnu/ -Wall -Wno-pointer-sign -Wno-unused-but-set-variable -O3 *.c -lgcrypt -lgpg-error -lbz2 -lz -I. -o PGPCrack-NG
compress.c:34:18: fatal error: zlib.h: No such file or directory
#include <zlib.h>
^
compilation terminated.
iobuf.c: In function ‘iobuf_ioctl’:
iobuf.c:1569:7: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation]
if (DBG_IOBUF)
^~
iobuf.c:1573:2: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the ‘if’
if (!a && !intval && ptrval)
^~
openfile.c: In function ‘overwrite_filep’:
openfile.c:90:5: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation]
if( cpr_enabled () )
^~
openfile.c:92:2: note: ...this statement, but the latter is misleadingly indented as if it is guarded by the ‘if’
return 1;
^~~~~~
Makefile:2: recipe for target 'target' failed
make: *** [target] Error 1
I have the following zlib files, so I'm not sure why it's not finding them.
linux-headers-4.9.0-4-amd64: /usr/src/linux-headers-4.9.0-4-amd64/include/config/jffs2/zlib.h
linux-headers-4.9.0-4-amd64: /usr/src/linux-headers-4.9.0-4-amd64/include/config/pstore/zlib/compress.h
linux-headers-4.9.0-4-amd64: /usr/src/linux-headers-4.9.0-4-amd64/include/config/squashfs/zlib.h
linux-headers-4.9.0-4-amd64: /usr/src/linux-headers-4.9.0-4-amd64/include/config/ubifs/fs/zlib.h
linux-headers-4.9.0-4-amd64: /usr/src/linux-headers-4.9.0-4-amd64/include/config/zlib/deflate.h
linux-headers-4.9.0-4-amd64: /usr/src/linux-headers-4.9.0-4-amd64/include/config/zlib/inflate.h
I have the following zlib files, so I'm not sure why it's not finding them. ... linux-headers-4.9.0-4-amd64: /usr/src/linux-headers-4.9.0-4-amd64/include/config/zlib/deflate.h
These are kernel headers!
Just install the zlib-dev
or a similar package to build the package on Debian. IIRC latest Debian breaks the existing project documentation.
[gpg2john] MDC is missing, expect lots of false positives!
This is the real problem and can't be solved 100%. It also has nothing to do with the build problems you report.
Thanks. I installed zlib1g-dev
and it built. (zlib-dev
wasn't available). You might want to add that to the readme.
I don't have the resources to maintain documentation for every new distribution. You may send patches if you care.
I'm a github novice but i'll try.
The next problem is the readme shows:
3. john -i -stdout | ./PGPCrack-NG <PGP file>
and john
gives Unknown option: "-i"
. Is that meant to be for incremental mode, despite using a word list?
Read the updated documentation. Things evolve, and existing documentation breaks all the time. I don't have the resources to offer you any further support.
Some time ago, there was only one john option starting with letter i, --incremental
, now jumbo has
--incremental --input-encoding= --internal-codepage=
That's why, -i is ambiguous. The error message could be more helpful in such a case.
Use -inc
or --inc
or --incremental
...
Setup
OK cases
Not OK
GPG files lacking MDC seem to be run into parsing problems.
If the size and contents of "large" file are varied, gpg2john runs into multiple different problems. These
gpg2john
error message can also change for different runs ofgpg
on the same input file.For other files, the following error message is shown by
gpg2john
,Note: Cracking of symmetrically encrypted files lacking MDC is currently not practically possible due to too many false positives involved. See https://github.com/magnumripper/JohnTheRipper/issues/1892 for details.