Add zip support info to --list=build-info

[claudioandre-br]

See https://www.openwall.com/lists/john-users/2019/02/09/1

Something simple. E,g,:

#if HAVE_LIBZ
    printf("pkzip support: yes")
#else
    printf("pkzip support: no")
#endif

[claudioandre-br]

$ john test.hash 
Using default input encoding: UTF-8
Loaded 1 password hash (PKZIP [32/64])
Will run 2 OpenMP threads
Proceeding with single, rules:Single
Press 'q' or Ctrl-C to abort, almost any other key for status
Warning: Only 3 candidates buffered for the current salt, minimum 8
needed for performance.
Warning: Only 6 candidates buffered for the current salt, minimum 8
needed for performance.
Warning: Only 5 candidates buffered for the current salt, minimum 8
needed for performance.
Warning: Only 4 candidates buffered for the current salt, minimum 8
needed for performance.
Warning: Only 2 candidates buffered for the current salt, minimum 8
needed for performance.
Almost done: Processing the remaining buffered candidate passwords, if any
Warning: Only 6 candidates buffered for the current salt, minimum 8
needed for performance.
Proceeding with wordlist:/snap/john-the-ripper/current/run/password.lst, rules:Wordlist
12345            (test.zip)
1g 0:00:00:00 DONE 2/3 (2019-02-09 20:23) 1.754g/s 97270p/s 97270c/s 97270C/s 123456..Peter
Use the "--show" option to display all of the cracked passwords reliably
Session completed

$ cat test.hash 
test.zip:$pkzip2$3*1*1*0*8*24*7cf7*7f0b*60f3dd0871b7f9d6a5c6b248bee749e90dfa0c380bc3fc2f5cb7e1c548a737e7fd497fea*1*0*8*24*862d*7ef8*84c49f1408fb2c5e4eb7e08dfbde4bea5be6016ab505af43ad0779efa1e6ab1ed8c4d3e7*2*0*4f3*c46*f5f912f0*537*23*8*4f3*f5f9*7f07*b92aac438c4df3217c536db80d7db01388936af6c1f5b41c38de103d3b29902d1be3ed30c195f6b31b9f81835046df95c5c2a6b21e4069acc2dec9fa3e64cb9fafa4ceab289e91578cf202a5eb7e0cdfe6a20523a0fca4964ef159d2da21aac9249858061392fda2e274a6baa0c7934b66cf62f076bbe06c2f772ebde5f1ba986815478eb5e9544dd8cf1172d2395bbd8929caa3f7b3d075b93a6493a26f9f860b542611942ebbc7767e96fc0385ba3efc2a7f10a273beb72d45e743d8e356d9dcc3c150f0caccab53f727562ee231dd856e2d8725c5c715bddf5a3198024f2616f611af5e2da3b7af51dea5192bfd572498eab45933d1620ebc4c2b9b0f4eae13f793e9832634b8c1347d41f65f611b1cac23c748a056133c165262777956e1d8d2d272eefd841494418db863d8331d6bf66f47a8ee7727e7a3adb9f5e079214ecf88034a083f1addbe8a1836ff239878893a0ad68d897ed2977bdddbaada1cbf57d05d3ce08026349e4496be99a027acdc8bfedec18f456fbd31159a513fc842e85eb744a5d0f737678785297aeb9bca9f6b8ae7792f43e15b8009147f06b111d8c537aa20b7045ff91c488e4ee58b04bf50e0273124a11e204fc53d46c1adcd7d76865e70bdaebd66d8e38725eda5bb8521ea2f4c777d0ecf40585a9c6294bd7abf4c486dd9ca2a42985ee46661ea3147674900b55646324c29b8f1778d858a4df05bbb67f5a0ba5484fd1b6a7ea44e93a91a7350a4fc87972c228861cfcabc1651444a67675bc874a7870d433d06cdf7bf68e584409beb03b431b86b91814d3e397b091635429683168de0fcd5a43f58e6441a509389c25dccb58981cf603e9c9d2d772b95a333c2fee9b0a24a5cc0d86c503269e70fcc015af503d492ad9e4c372eb8e321d6626453f939318d66d0c4cf80cfd18152ab8946e20afd5750ea511d8813dedefba080ad43ed89787013ce717278e311cbb61355e291e2b31b3730407d23d5375aa5cf537f1d19f14564d882587bcf5e69ec4c0b558d39082a3ca0104bbafa56d9dbdeec1b9fccc463d12912d7a0b0ba719fea0481ff094bedaf0a640ffe6ca6df8dfc6985af9062e625791024d5980d526b3988fde9641c83c64ce9d7911a442a59c0354f83b5457a4129972907ec0d98bd8398e57396b35da0cab667abe6e144612f665ae57388f8556befd1409b60e709725425006636bff1c73c6bf9cfb1b592aabd47f8bc85765820b89a50eace6f9db95e27fdf74c3a319487819d7a9931047272896e1925f231c97a25e5c05765069c13216cb54e04998b46c901b2991e1049969d0f454739277fc013820cea9d0d857d016b37fafcc731cc0638dc7f51ae998b314d4f278454c2b71d61fe84201da4b7225651c87ed205bf1d36cdd2ade823209b02344f4e6115de0c8fb94f9dcfa06069d02c8aaf29923f10fdfee73b9db4ec34715911f09f7785fd578216f539448352362d09b27770f2b79694383b416ee1d16760555223532fa0ca649913b21050c3f45d5d0cc7622289d0cb69750cef7e58c1f5dd04a41f927b19cf8d3a5ac5e022491b33d0dc5c7679670d3dd510c4f67221cd143d1a6190cd3f2e91a43394bb135a214864d69ff0818e014ab23eae8a0994977e048626f96c61f09f23a5e66d868ed58e7864f1a1d020d291c799d9c021a4892d2d55496d943ed977c69160f672a3aad12679322aefae97b808c28e1b42ee48a061c9a6a392bf855ac107d09b0a54fdaf4208a9ba3d0c0c28683373a277e5c35da364acce*$/pkzip2$::test.zip:b.txt, c.txt, a.txt:test.zip

[frank-dittrich]

If we do this, we should list all (CPU?) formats which have been disabled during build time because of some missing preconditions. So, instead of

dynamic_fmt.c:#warning Notice: Dynamic format disabled from build.
pkzip_fmt_plug.c:#warning pkzip format requires zlib to function. The format has been disabled
rar_fmt_plug.c:#warning ": target system requires aligned memory access, rar format disabled:"
stribog_fmt_plug.c:#warning Stribog-256 and Stribog-512 formats require SSE 4.1, formats disabled

we should use some other mechanism which emits the build time warning and also adds the warning to a list that can be printed with --list=build-info.

[solardiz]

As an alternative to introducing a new mechanism like Frank suggests, we could have in the build-info code an array with a list of format names for which we have compile-time exclusion logic and the code could search through all included formats for those names. I think this will be easier to add.

[frank-dittrich]

@solardiz Your solution is easier to add. But each future exclusion will require two changes, and thus is more error prone.

[jfoug]

Just have an optional 4th #define in each format. then the format will 'self-describe' itself as turned off. Right now, we a define to 'build' the format type, and format registers:

#if FMT_EXTERNS_H
extern struct fmt_main fmt_sevenzip;
#elif FMT_REGISTERS_H
john_register_one(&fmt_sevenzip);
#else
....

We then build fmt_externs.h and fmt_register.h What I propose is we have another 'branch' (i.e. another #elif SOMETHING before the , or even wrap the ENTIRE thing in it's own #if (for disabled)

So we have something like:

#if some_condition_why_we_are_turning_off
#define TURN_ME_OFF 1
#endif

#if FMT_DISABLED_H && TURN_ME_OFF
john_register_disabled("seven-zip");
#elif FMT_EXTERNS_H
extern struct fmt_main fmt_sevenzip;
#elif FMT_REGISTERS_H
john_register_one(&fmt_sevenzip);
#elif !TURN_ME_OFF 
///... real format code

If a format never gets disabled

#if FMT_DISABLED_H
// we never disable this format.
#elif FMT_EXTERNS_H
extern struct fmt_main fmt_sevenzip;
#elif FMT_REGISTERS_H
john_register_one(&fmt_sevenzip);
#elif
///... real format code

Then we simply build a 3rd header, calling gcc passing in -DFMT_DISABLEDH=1 the way the other 2 fmt*.h currently are built. This new cpp build will build is a fmt_disabled.h. Then we properly include that header file wherever it is needed, and only formats which were disabled during the build will have code in that include (which could be zero formats being disabled). I am pretty sure this is a pretty trivial solution. A LOT of code changes, but should be pretty easy to do.

[solardiz]

Here's an alternative to Jim's idea: add FMT_DISABLED flag and set it on formats that are compile-time disabled, excluding their actual implementation but including a dummy struct fmt_main in their name. Then some other parts of JtR would skip those formats when scanning the formats' list, but --list=build-info would print something like "Disabled formats: " followed by a loop printing the names of each format it finds having FMT_DISABLED on it.

Alternatively, we could have NULL for a key format method like init indicate that the format is disabled, without introducing a new flag.

Additionally, we could also provide almost-real valid() even for those disabled formats, and have it print a warning (just once) if the supplied hash matches the format, then return 0. I think this would actually help many users, unlike a mere addition to --list=build-info which only a few would bother using. We would also be able to make those messages from valid() more verbose and more specific (mention the specific missing library, etc. as the reason why the format is disabled).

[jfoug]

Additionally, we could also provide almost-real valid() even for those disabled formats, and have it print a warning (just once) if the supplied hash matches the format, then return 0. I think this would actually help many users, unlike a mere addition to --list=build-info which only a few would bother using. We would also be able to make those messages from valid() more verbose and more specific (mention the specific missing library, etc. as the reason why the format is disabled).

I think this is the BEST winning solution. Here, we would have to build the format record 'smartly', linking in either NULL, or something (some fmt_default() functions), if the format is disabled. The valid() would NEVER be disabled, although there may be 2 of those functions (one real one, and one 'stripped down',

I just tossed out a real quick 'off the cuff' reply. I think this valid() printing 'format has been disabled', is also off the cuff, BUT I think this is really on the right path.

It would make writing a format which 'can' be disabled a bit more tricky, BUT I do see the payoff may be much better, once it is done, AND it may be able to be done without ANY changes to the format structure (however, the format structure 'logic' version might be a bit different, i.e. adding a flag, having a special method pointer being null, etc). But within writing the code to disable a format, we can now easily describe TO the user, WHEN they are running john, exactly what/why this is not working for them, along with steps needed to get the job done (i.e. install gmp-dev, before doing a configure etc). Good stuff.

[frank-dittrich]

People using pre-compiled binaries then wouldn't know why a format has been "disabled". (Not that this knowledge would be of much help for them.)

We must make sure users cannot confuse such compile-time disabled formats with formats disabled by config file settings.

And I wouldn't want a ton of "disabled" warnings if I use ./john without --format=. (May be only print these warnings if no format found any valid hashes.)

[solardiz]

We can word the messages such that they're informative to people using pre-compiled binaries as well.