Add a new --list=rules-details option(s)

[jfoug]

(Hopefully, this is not already available, making this a PEBKAC which I now see has been renamed invalid, lol. If we were to replace it, I much prefer using the "ID:107 error" moniker myself, lol.)

• [ ] --list=rules-details
• [x] --list=rules-details-noprep (handled by -list=list-data:List.Rules:rule_name)
• [ ] --list=rules-details-raw (NOTE, likely better to add as --list=list-data-raw:name, vs just allowing List.Rules:rulename lists to be dumped raw)

This option will dump out a nice display of the rules which make up a section. It will be similar to the output we see when logging is enabled, when a new rule is found, then preprocessed and accepted.

So output would be something like this:

$ ./john -list=rules-details  --rules=single

dump of processed rules from [List.Rules=single]
...
Rule #560: '-c l Q T4 Q M T4 Q l Tm Q C Q u Q X0z0 'l' accepted as 'lQT4QMT4QlTmQCQuQX0z0'l'
Rule #561: '-c l Q T5 Q M T5 Q l Tm Q C Q u Q X0z0 'l' accepted as 'lQT5QMT5QlTmQCQuQX0z0'l'
Rule #562: '-c l Q T6 Q M T6 Q l Tm Q C Q u Q X0z0 'l' accepted as 'lQT6QMT6QlTmQCQuQX0z0'l'
Rule #563: '-c l Q T7 Q M T7 Q l Tm Q C Q u Q X0z0 'l' accepted as 'lQT7QMT7QlTmQCQuQX0z0'l'
Rule #564: '-c l Q T8 Q M T8 Q l Tm Q C Q u Q X0z0 'l' accepted as 'lQT8QMT8QlTmQCQuQX0z0'l'
Rule #565: '-c l Q T9 Q M T9 Q l Tm Q C Q u Q X0z0 'l' accepted as 'lQT9QMT9QlTmQCQuQX0z0'l'
Rule #566: '-c l Q TA Q M TA Q l Tm Q C Q u Q X0z0 'l' accepted as 'lQTAQMTAQlTmQCQuQX0z0'l'
Rule #567: '-c l Q TB Q M TB Q l Tm Q C Q u Q X0z0 'l' accepted as 'lQTBQMTBQlTmQCQuQX0z0'l'
Rule #568: '-c l Q TC Q M TC Q l Tm Q C Q u Q X0z0 'l' accepted as 'lQTCQMTCQlTmQCQuQX0z0'l'
Rule #569: '-c l Q TD Q M TD Q l Tm Q C Q u Q X0z0 'l' accepted as 'lQTDQMTDQlTmQCQuQX0z0'l'
Rule #570: '-c l Q TE Q M TE Q l Tm Q C Q u Q X0z0 'l' accepted as 'lQTEQMTEQlTmQCQuQX0z0'l'
Rule #571: '-c >2 <G %2?a l T0 M T2 T4 T6 T8 TA TC TE Q M l Tm Q X0z0 'l' accepted as '>2<G%2?alT0MT2T4T6T8TATCTEQMlTmQX0z0'l'
Rule #572: '-c >2 <G %2?a u T0 M T2 T4 T6 T8 TA TC TE Q M l Tm Q X0z0 'l' accepted as '>2<G%2?auT0MT2T4T6T8TATCTEQMlTmQX0z0'l'
...

There could be a 2nd 'feature', say --list=rules-details-noprep which would skip the preprocessor. So you would see something like:

$ ./john -list=rules-details-noprep  --rules=single

dump of processed rules from [List.Rules=single] without pre-processing
...
Rule #43: -c l Q T[1-9A-E] Q M T\0 Q l Tm Q C Q u Q X0z0 'l
Rule #44: -c >2 <G %2?a [lu] T0 M T2 T4 T6 T8 TA TC TE Q M l Tm Q X0z0 'l
...

There might even be a 3rd option (Not sure about this, since you can simply vi the conf file) This option (something like --list=rules-details-original) would simply dump the section out of the .conf file intact, commenting, include directives, etc intact. This may be much harder to do, since we do not know which conf file things are in. Also, we may have to pre-process conf files expanding includes. Since we are wanting to output 'exactly' what is in the ruleset, we would not be dropping comments, AND probably should make best effort to also include any prelim comments. Likely this will be 100% custom code, we can not use any of the config handling code (unless we 'adjust' it some, I think the comment inclusion is the biggest hurdle, but I am not sure).

Example:

$ ./john -list=rules-details-raw  --rules=all

dump raw rules section from [List.Rules=all]

# Everything, including all KoreLogic and the rest of included hashcat rules.
# Only for very fast hashes and/or Single mode. Some of these rules are of
# ridiculous quality and lack optimizations - you have been warned.
[List.Rules:All]
.include [List.Rules:Jumbo]
.include [List.Rules:KoreLogic]
.include [List.Rules:T9]
.include [List.Rules:hashcat]

[jfoug]

Note, this does 'some' of what is listed here:

--list=rules-details-noprep --rules=single It could probably 'work' for the -list=rules-details --rules=single method. It does not show the rule number (not really important), but the rest is done (i.e file and section inclusion). So for all purpose, --list=rules-details-noprep is completed.

[magnumripper]

I like this!

[jfoug]

The 3rd 'option' is the harder one. This will require some 'thought'. But this would be the easy way to dump out a section (with comments above, and internal to that section).

For item 3 (Off the cuff, I am not looking at code, and some of this may be there), I would think this type algorithm.

1. create an list of conf lines. It would be best if this list had a next and prev pointer (so we can backup and grab comments, BUT we can use a next only list, by walking it twice)
2. read john.conf. In reading, we do not remove comments (blanks, ???).
3. inline expand includes EXCEPT for .include [section] We leave those section include LINES in the list.
4. to dump a section, we walk the list to find the section. Then walk backwards, until we find a non-blank line that is not a comment (or start of list, if this is the first section). That last comment line is the START of the section.
5. We then walk the pointer to the section name forward until we hit end of list (i.e. last section) OR we hit a new section. If we hit a new section, back up this tail pointer until we hit the first non-blank, non comment line (i.e. the trailing comment goes with next section).
6. We now have the start and end of what we want to output, so we output and exit gracefully.

The biggest unknown is being able to read the config, keeping comments (possibly blank lines), and not expanding the 'section' includes. It may be easy to do this, by creating a function in config that sets a variable (signaling us to use a bit different behavior), then dump and re-read the config, using this new flag variable set. The config data would be 'not' valid, but since we are exiting once we dump the data, it is not that bad of a situation (IMHO).

This change really should be pretty trivial (I think). The most code change will likely be the plumbing to get it working (options, reading/setting/handling, the bash completion, updating dox, etc).

[jfoug]

Actually, it may be better to 'default' to the raw. So --list=rules-details would show the un-preprocessed rules, and comments. Then we could either build a --list=rules-details-pp to dump pre-processed rules, OR simly use the existing --list=list-data:List.Rules:rule_name logic (even having --list=rules-details-pp simply be a alias for the list-data option.

What do you think about this implementation?

(Note, it may be 'best' to build it this way): real function: --list=list-data-raw:List.Rules:rule_name alias: --list=rules-details=rule_name

real function: --list=list-data:List.Rules:rule_name alias: --list=rules-details-pp=rule_name

That way we get ability to raw dump any list, and simply add the 'helper' alias for rules, and we may decide we do not really 'need' the aliases. That the list-data-raw and list-data are sufficient.

[frank-dittrich]

(Hopefully, this is not already available, making this a PEBKAC which I now see has been renamed invalid, lol. If we were to replace it, I much prefer using the "ID:107 error" moniker myself, lol.)

No. Currently we only have --list=rules listing all the rules sections and --list=list-data:list.rules:single listing the contents of the [List.Rules:Single] section (resolving included sections and files, but not expanding to individual rules.

--list=rules-details --list=rules-details-noprep (handled by -list=list-data:List.Rules:rule_name)

Instead of

$ ./john --list=rules-details  --rules=single

I'd prefer

$ ./john --list=rules-details:single

[magnumripper]

I agree with Frank (and I did not even remember we already have some of this 😆 )

[jfoug]

Instead of

$ ./john --list=rules-details  --rules=single

I'd prefer

$ ./john --list=rules-details:single

I highly agree to this. Much, much better syntax.

[frank-dittrich]

read john.conf. In reading, we do not remove comments (blanks, ???). Unless the command line has a --config= option

[frank-dittrich]

(and I did not even remember we already have some of this :laughing: )

I remember, because I implemented it. (The idea was to automate distributed cracking, by generating a set of tiny config files and other command line options, and idle clients should be able to pick up the next package ...)