JtR help screen (showing formats), is simply too long

[jfoug]

Here is the current output

$ ../run/john
John the Ripper password cracker, version 1.8.0.2-jumbo-1-bleeding_omp [linux-gnu 64-bit SSE4.1-autoconf]
Copyright (c) 1996-2014 by Solar Designer and others
Homepage: http://www.openwall.com/john/

Usage: john [OPTIONS] [PASSWORD-FILES]
--single[=SECTION]        "single crack" mode
--wordlist[=FILE] --stdin wordlist mode, read words from FILE or stdin
                  --pipe  like --stdin, but bulk reads, and allows rules
--loopback[=FILE]         like --wordlist, but fetch words from a .pot file
--dupe-suppression        suppress all dupes in wordlist (and force preload)
--prince[=FILE]           PRINCE mode, read words from FILE
--encoding=NAME           input encoding (eg. UTF-8, ISO-8859-1). See also
                          doc/ENCODING and --list=hidden-options.
--rules[=SECTION]         enable word mangling rules for wordlist modes
--incremental[=MODE]      "incremental" mode [using section MODE]
--mask=MASK               mask mode using MASK
--markov[=OPTIONS]        "Markov" mode (see doc/MARKOV)
--external=MODE           external mode or word filter
--stdout[=LENGTH]         just output candidate passwords [cut at LENGTH]
--restore[=NAME]          restore an interrupted session [called NAME]
--session=NAME            give a new session the NAME
--status[=NAME]           print status of a session [called NAME]
--make-charset=FILE       make a charset file. It will be overwritten
--show[=LEFT]             show cracked passwords [if =LEFT, then uncracked]
--test[=TIME]             run tests and benchmarks for TIME seconds each
--users=[-]LOGIN|UID[,..] [do not] load this (these) user(s) only
--groups=[-]GID[,..]      load users [not] of this (these) group(s) only
--shells=[-]SHELL[,..]    load users with[out] this (these) shell(s) only
--salts=[-]COUNT[:MAX]    load salts with[out] COUNT [to MAX] hashes
--save-memory=LEVEL       enable memory saving, at LEVEL 1..3
--node=MIN[-MAX]/TOTAL    this node's number range out of TOTAL count
--fork=N                  fork N processes
--pot=NAME                pot file to use
--list=WHAT               list capabilities, see --list=help or doc/OPTIONS
--format=NAME             force hash type NAME: 7z AFS agilekeychain aix-smd5
                          aix-ssha1 aix-ssha256 aix-ssha512 asa-md5 bcrypt
                          bfegg Bitcoin blackberry-es10 Blockchain bsdicrypt
                          chap Citrix_NS10 Clipperz cloudkeychain cq CRC32
                          crypt dahua descrypt Django django-scrypt dmd5 dmg
                          dominosec dragonfly3-32 dragonfly3-64 dragonfly4-32
                          dragonfly4-64 Drupal7 dummy dynamic_n eCryptfs EFS
                          eigrp EncFS EPI EPiServer fde FormSpring Fortigate
                          gost gpg HAVAL-128-4 HAVAL-256-3 hdaa HMAC-MD5
                          HMAC-SHA1 HMAC-SHA224 HMAC-SHA256 HMAC-SHA384
                          HMAC-SHA512 hMailServer hsrp IKE ipb2 KeePass
                          keychain keyring keystore known_hosts krb4 krb5
                          krb5-18 krb5pa-md5 krb5pa-sha1 kwallet LastPass LM
                          lotus5 lotus85 lp LUKS MD2 md4-gen md5crypt md5ns
                          mdc2 MediaWiki MongoDB Mozilla mscash mscash2
                          MSCHAPv2 mschapv2-naive mssql mssql05 mssql12 mysql
                          mysql-sha1 mysqlna net-md5 net-sha1 nethalflm netlm
                          netlmv2 netntlm netntlm-naive netntlmv2 nk nsldap NT
                          nt2 o5logon ODF Office oldoffice OpenBSD-SoftRAID
                          openssl-enc OpenVMS oracle oracle11 osc Panama
                          PBKDF2-HMAC-SHA1 PBKDF2-HMAC-SHA256
                          PBKDF2-HMAC-SHA512 PDF PFX phpass PHPS pix-md5 PKZIP
                          po postgres PST PuTTY pwsafe RACF RAdmin RAKP rar
                          RAR5 Raw-Blake2 Raw-Keccak Raw-Keccak-256 Raw-MD4
                          Raw-MD5 Raw-MD5u Raw-SHA Raw-SHA1 Raw-SHA1-Linkedin
                          Raw-SHA1-ng Raw-SHA224 Raw-SHA256 Raw-SHA256-ng
                          Raw-SHA384 Raw-SHA512 Raw-SHA512-ng ripemd-128
                          ripemd-160 rsvp Salted-SHA1 sapb sapg saph scrypt
                          sha1-gen sha1crypt sha256crypt sha512crypt Siemens-S7
                          SIP skein-256 skein-512 skey Snefru-128 Snefru-256
                          SSH SSH-ng SSHA512 STRIP SunMD5 sxc Sybase-PROP
                          sybasease tc_aes_xts tc_ripemd160 tc_sha512
                          tc_whirlpool tcp-md5 Tiger tripcode VNC vtp wbb3
                          whirlpool whirlpool0 whirlpool1 WoWSRP wpapsk xsha
                          xsha512 ZIP

How about this type change?

John the Ripper password cracker, version 1.8.0.2-jumbo-1-bleeding [cygwin 32-bit AVX-autoconf]
Copyright (c) 1996-2014 by Solar Designer and others
Homepage: http://www.openwall.com/john/

Usage: jtr_vc [OPTIONS] [PASSWORD-FILES]
--single[=SECTION]        "single crack" mode
--wordlist[=FILE] --stdin wordlist mode, read words from FILE or stdin
                  --pipe  like --stdin, but bulk reads, and allows rules
--loopback[=FILE]         like --wordlist, but fetch words from a .pot file
--dupe-suppression        suppress all dupes in wordlist (and force preload)
--encoding=NAME           input encoding (eg. UTF-8, ISO-8859-1). See also
                          doc/ENCODING and --list=hidden-options.
--rules[=SECTION]         enable word mangling rules for wordlist modes
--incremental[=MODE]      "incremental" mode [using section MODE]
--mask=MASK               mask mode using MASK
--markov[=OPTIONS]        "Markov" mode (see doc/MARKOV)
--external=MODE           external mode or word filter
--regex=REGEXPR           regular expression mode (see doc/README.librexgen)
--stdout[=LENGTH]         just output candidate passwords [cut at LENGTH]
--restore[=NAME]          restore an interrupted session [called NAME]
--session=NAME            give a new session the NAME
--status[=NAME]           print status of a session [called NAME]
--make-charset=FILE       make a charset file. It will be overwritten
--show[=LEFT]             show cracked passwords [if =LEFT, then uncracked]
--test[=TIME]             run tests and benchmarks for TIME seconds each
--users=[-]LOGIN|UID[,..] [do not] load this (these) user(s) only
--groups=[-]GID[,..]      load users [not] of this (these) group(s) only
--shells=[-]SHELL[,..]    load users with[out] this (these) shell(s) only
--salts=[-]COUNT[:MAX]    load salts with[out] COUNT [to MAX] hashes
--save-memory=LEVEL       enable memory saving, at LEVEL 1..3
--node=MIN[-MAX]/TOTAL    this node's number range out of TOTAL count
--pot=NAME                pot file to use
--list=WHAT               list capabilities, see --list=help or doc/OPTIONS
--format=NAME             force hash type NAME can be one of the following:

    7z AFS agilekeychain aix-smd5 aix-ssha1 aix-ssha256 aix-ssha512 asa-md5
    bcrypt bfegg blackberry-es10 Blockchain bsdicrypt chap Citrix_NS10 Clipperz
    cloudkeychain cq CRC32 dahua descrypt Django django-scrypt dmd5 dmg
    dominosec dragonfly3-32 dragonfly3-64 dragonfly4-32 dragonfly4-64 Drupal7
    dummy dynamic_n eCryptfs EFS eigrp EncFS EPI EPiServer fde FormSpring
    Fortigate gost gpg HAVAL-128-4 HAVAL-256-3 hdaa HMAC-SHA224 HMAC-SHA256
    HMAC-SHA384 HMAC-SHA512 hMailServer hsrp IKE ipb2 KeePass keychain keyring
    keystore krb4 krb5 krb5-18 krb5pa-md5 krb5pa-sha1 kwallet LastPass LM
    lotus5 lotus85 LUKS MD2 md4-gen md5crypt md5ns MediaWiki MongoDB Mozilla
    mscash mscash2 MSCHAPv2 mschapv2-naive mssql mssql05 mssql12 mysql
    mysql-sha1 mysqlna net-md5 net-sha1 nethalflm netlm netlmv2 netntlm
    netntlm-naive netntlmv2 nk nsldap nt2 o5logon ODF Office oldoffice
    openssl-enc oracle oracle11 osc Panama PBKDF2-HMAC-SHA1 PBKDF2-HMAC-SHA256
    PBKDF2-HMAC-SHA512 PDF PFX phpass PHPS pix-md5 PKZIP po postgres PST PuTTY
    pwsafe RACF RAdmin RAKP rar RAR5 Raw-Blake2 Raw-Keccak Raw-Keccak-256
    Raw-MD4 Raw-MD5 Raw-MD5u Raw-SHA1 Raw-SHA1-Linkedin Raw-SHA224 Raw-SHA256
    Raw-SHA384 Raw-SHA512 ripemd-128 ripemd-160 rsvp Salted-SHA1 sapb sapg
    scrypt sha1-gen sha1crypt sha256crypt sha512crypt Siemens-S7 SIP skein-256
    skein-512 skey Snefru-128 Snefru-256 SSH SSH-ng SSHA512 STRIP SunMD5 sxc
    Sybase-PROP sybasease tc_aes_xts tc_ripemd160 tc_sha512 tc_whirlpool
    tcp-md5 Tiger tripcode VNC vtp wbb3 whirlpool whirlpool0 whirlpool1 WoWSRP
    xsha xsha512

[jfoug]

Here is all it takes (pretty trivial stuff). I like this fatter list of formats better. BUT is there any reason why this change CAN NOT be made??

$ git diff options.c
diff --git a/src/options.c b/src/options.c
index e012507..782afab 100644
--- a/src/options.c
+++ b/src/options.c
@@ -343,10 +343,11 @@ JOHN_USAGE_FORK \
 "--list=WHAT               list capabilities, see --list=help or doc/OPTIONS\n"

 #define JOHN_USAGE_FORMAT \
-"--format=NAME             force hash type NAME:"
+"--format=NAME             force hash type NAME can be one of the following:\n\n" \
+       JOHN_USAGE_INDENT

 #define JOHN_USAGE_INDENT \
-"                         " // formats are prepended with a space
+"   " // formats are prepended with a space

 #if defined(HAVE_OPENCL) && defined(HAVE_CUDA)
 #define JOHN_USAGE_GPU \
@@ -402,7 +403,7 @@ static void print_usage(char *name)
        printf("%s", JOHN_USAGE_GPU);
 #endif
        printf("%s", JOHN_USAGE_FORMAT);
-       column = sizeof(JOHN_USAGE_FORMAT);
+       column = strlen(JOHN_USAGE_INDENT);

        i = 0;
        do {
@@ -417,6 +418,7 @@ static void print_usage(char *name)
                printf(" %s%s", label, formats_list[i] ? "" : "\n");
        } while (formats_list[i]);
        MEM_FREE(formats_list);
+       printf ("\n");

        exit(0);
 }

[magnumripper]

Actually I think we should instead separate the format list from the usage blob. Just skip the formats list and instead print a last line saying you should use --list=formats to see it. And that list is already without indentation.

Would that break TS though? And/or bash completion?

[jfoug]

Ok, do you want me to make this change (NOTE, i have to make a change to the TS, which I have also done).

$ ../run/john
John the Ripper password cracker, version 1.8.0.2-jumbo-1-bleeding [cygwin 64-bit AVX-autoconf]
Copyright (c) 1996-2014 by Solar Designer and others
Homepage: http://www.openwall.com/john/

Usage: john [OPTIONS] [PASSWORD-FILES]
--single[=SECTION]        "single crack" mode
--wordlist[=FILE] --stdin wordlist mode, read words from FILE or stdin
                  --pipe  like --stdin, but bulk reads, and allows rules
--loopback[=FILE]         like --wordlist, but fetch words from a .pot file
--dupe-suppression        suppress all dupes in wordlist (and force preload)
--prince[=FILE]           PRINCE mode, read words from FILE
--encoding=NAME           input encoding (eg. UTF-8, ISO-8859-1). See also
                          doc/ENCODING and --list=hidden-options.
--rules[=SECTION]         enable word mangling rules for wordlist modes
--incremental[=MODE]      "incremental" mode [using section MODE]
--mask=MASK               mask mode using MASK
--markov[=OPTIONS]        "Markov" mode (see doc/MARKOV)
--external=MODE           external mode or word filter
--stdout[=LENGTH]         just output candidate passwords [cut at LENGTH]
--restore[=NAME]          restore an interrupted session [called NAME]
--session=NAME            give a new session the NAME
--status[=NAME]           print status of a session [called NAME]
--make-charset=FILE       make a charset file. It will be overwritten
--show[=LEFT]             show cracked passwords [if =LEFT, then uncracked]
--test[=TIME]             run tests and benchmarks for TIME seconds each
--users=[-]LOGIN|UID[,..] [do not] load this (these) user(s) only
--groups=[-]GID[,..]      load users [not] of this (these) group(s) only
--shells=[-]SHELL[,..]    load users with[out] this (these) shell(s) only
--salts=[-]COUNT[:MAX]    load salts with[out] COUNT [to MAX] hashes
--save-memory=LEVEL       enable memory saving, at LEVEL 1..3
--node=MIN[-MAX]/TOTAL    this node's number range out of TOTAL count
--fork=N                  fork N processes
--pot=NAME                pot file to use
--list=WHAT               list capabilities, see --list=help or doc/OPTIONS
--format=NAME             force hash of type NAME. The supported hashes can
                          be seen with --list=formats and --list=subformats

[jfoug]

Would that break TS though? And/or bash completion?

TS, yes. But I have fixed it. https://github.com/magnumripper/jtrTestSuite/commit/70cd598a3d857ed829beaec56f93ca28bc3c62ad I just assume that if I do not find a ':' character on the line starting with --format=NAME, then it is this new format, and simply get the data using --list=formats and have slightly modified regex to parse it.

As for bash completion, I bet this will break it also, but the change there likely will be trivial also (unless Frank already is using the --list=formats)

@frank-dittrich can you comment. I really do not want to check this in, when there are known parts of the toolset that will fail.

[frank-dittrich]

Currently, john.bash_completion tries to process john's usage output to get the format list (line 335) and adds the format classes mentioned in ./john --list=hidden-options --format=CLASS output. (line 338)

It ould have to be changed to use --list=formats (or --list=format-details instad of usage output, and process the usage output only as a fallback (for core) if --list=formats doesn't succeed.

[jfoug]

@frank-dittrich Please review 245aeb2 I am pretty sure this fixed the problem. NOTE, I have not checked in the new version yet, but with this change I will do so. I do not have bash completion set on my normal runtime (simply WAY too slow and gets in the way), so I really do not test this much.

[jfoug]

9e3bb75 is the update to john usage screen, removing the format strings.

[magnumripper]

Looks good to me, though I think I would prefer "The supported formats can be seen..." over "The supported hashes can be seen...".

I do not have bash completion set on my normal runtime (simply WAY too slow and gets in the way)

How come it's slow? The bash completion is the best improvement of all in Jumbo over the last 10 years. I would go mad without it.

[jfoug]

When I use it, hit tab, etc, it hangs, from 2 to 5s. I simply can type in 10x faster, and know what I want to say 90% of the time. Yes, when doing some obscure stuff I do not normally do, it would be faster, but I just feel it gets in my way.

[magnumripper]

That's in Cygwin only, right? We should find out the cause. You really can't notice any delay on OSX or Linux.

[jfoug]

cygwin, yes. Cygwin spawning is very slow. Compile is about 4x slower than on my ubuntu VM. ./configure is about 10x slower. I think that part of the problem is I have all sorts of old network mounts (we are talking OLD, some even running Novell, possibly DOS systems). There are also probably 10k files on my search path. I really need to clean that up from my cygwin path. Unfortunately things like c:\windows\system32 need to be in search path, and they are a graveyard of crap, 1000's of files, most not directly executable, but there are a few really important ones.

Ok, here is the current verbiage:

--format=NAME             force hash of type NAME. The supported hashes can
                          be seen with --list=formats and --list=subformats

What EXACTLY do you want that to be?

[magnumripper]

I thought this would look better but you can talk me out of it

--format=NAME             force hash of type NAME. The supported formats can
                          be seen with --list=formats and --list=subformats

[jfoug]

That works, I can easlily change it. The only thing I want to make sure is that '--list=formats' shows up on the bash-completion. I key off that search string. If it shows up in the format list, then I assume this version of john does not show the format list on the normal usage, and re-run to get a --list=formats string.