Rework cmd-line interface with Inclusions/eXclusions

[ankostis]

I'm opening this issue to discuss some proposed changes to the command-line options and behavior.

Hashing of strings arguments:

After #7, the omnihash command can consume strings from stdin, so now the "-s sting in cmd-line option" or failling-back to "assume args as strings" it is better to go:

• for simplification: currently the command behaves like echo and cat combined;

• it is too resilient(!), violating the "fail early" rule. Let's take this example:

  $ omnihash "`which foo`"
  Hashing string ''..
  DSA:                   da39a3ee5e6b4b0d3255bfef95601890afd80709
  ...

  That's the empty string DSA, but unless you peek at the informational heder or know it by hurt, it is easy to miss that foo file were not actually found, and accept only this:

  echo hello |  omniparse

  Add Inclusion/Exclusion flags:

Assuming #9 gets merged, unforeseen algos may clutter output - even now, it would be handier to eliminate some algos. So I'm suggesting to replace -c with one or more of the following:

• -X exclude: e.g. omnihash -X crc excludes all algos having this upper string in their names
• -I include: e.g. `omnihash -I sha3 prints only SHA3 algos. exlusions must apply afterwards,

[Miserlou]

There are two changes here, I don't like the first because my most common use case is usually to see if the thing I'm reversing is a file name or the file itself, so I want to be able to oh file and oh file -s. Now that we have stdin support, if you want to do stuff like you describe, you'd always do it that way anyway: which foo | oh ex.

No need to replace -c, but I'd like to have '-f' for filter or family for includes, so that I could only do the SHA family as you descibed, ex $ oh test -f sha. No capital flags, I hate those.

Similarly, I'd like to have a '-m' for match, so that I could either only show algos that match a target, or just highlight target out put. ex,

$ oh test -m a94a8fe5ccb19ba61c4c0873d391e987982fbbd3
Hashing string 'test'..
  SHA1:                  a94a8fe5ccb19ba61c4c0873d391e987982fbbd3
$ oh test -m a94a
Hashing string 'test'..
  SHA1:                  a94a8fe5ccb19ba61c4c0873d391e987982fbbd3
$ oh derp -m a94a8fe5ccb19ba61c4c0873d391e987982fbbd3
Hashing string 'test'..
No match found.

This is becoming a quite useful hash multitool!

[ankostis]

• `-m is very useful indeed!
• So -f is for inclusions, correct? By why not support both inclusion & exclusions?
• ...and here it comes a demand for a config-file ala setup.cfg, to be able to define the working-set of algorithms you desire....

...indeed it does become quite useful multitool.

[Miserlou]

I'm fine with exclusions, as long as it's -x not -X 😄 , although I think -f will get more usage.

Let's hold off on the config file for right now. People always think they want a config file.. but they don't. They really don't. If somebody demands a specific usage for it.. well, PRs welcome.

[ankostis]

No more PRs from me, I'm done my duty, just discussion :-) There is still something I don't understand:

... my most common use case is usually to see if the thing I'm reversing is a file name or the file itself, so I want to be able to oh file and oh file -s.

So the failback to string if file is not found, is it still desired? Because if file does not exist, the tools takes the initiative to hash the filename as string, even without -s - that's the problematic "too resilient" behavior I was referring above.

Finally, if -x is exclusion, -f for inclusions is somehow unsymmetrical....

[Miserlou]

Yeah, I want to keep "failback", as you put it, to be the string. That's still been primary use case so far, just hashing random strings as needed. That's why I wrote the tool to begin with! :-P (I need to hash a word! oh word! Bam!) Being able to discern what the input type is (file, url, string) is a feature, not a bug.

Yeah, it does seem asymmetrical, but that doesn't bother me because '-i' usually means input, and I think I'd actually prefer it if '-f' meant "family", as that's how most cryptographers using this would probably think. I'd wager that the common use case here is that you just want the sha family, sha3 family, ripe family, etc.

[ankostis]

@Miserlou I really believe that this 'fallback-to-string" is a dangerous feature - at least have it as opt-in with a new option (inverse of -s).
For safety I always use a bash alias oh="omnihash -s", but I believe it should be the other way round:those who want this failback should use an alias.

[Miserlou]

Why do you say dangerous?

[ankostis]

The danger comes from the non-determinism; the result depends on the working-directory you are.

I make example-presentations in bash for git, and frequently I create sample files like this:

echo f1>f1

Then when I type oh f1, I'm not sure what's the output. I have always to look at my screen for what comes back, and sometime I get surprised, particularly when the contents of the files have been updated.

Besides, all standard unix command-tools do not behave like that. Look at sha1sum and md5sum commands; they are used for the same purposes, but behave deterministically.

[ankostis]

By the way, I really enjoy traditional behavior, where if you don't specify anything, the command just reads stdin, and your screen stucks, so you "get the message"; either you start immediately typing the string you want and then [Ctrl+D], or you retry something like that:

cat | oh

Maybe I'm too addicted to the unix way...

[Miserlou]

Thanks for the -f PR @ankostis! Added a small fix and pushed to PyPi..

[ankostis]

Oups! Neglected my own children: plugins :-)

Thanks!

[ankostis]

I think there is a pretty simple solution for having a command with the traditional unix-y behavior & deterministic hashing (string or file, not just any of them, depending on the existence of the file), AND keeping the existing "shortcut" behavior you like:

• Use the shorter oh cmd tool for the current "guessing" behavior, and
• retrofit the omnihash tool to behave exactly like sha1sum cmd: sum files or STDIN when no args given.

@Miserlou what do you think?