Mangling

[WebBreacher]

Pipal is awesome at searching for specific strings in passwords. Does the password have the word "pink"? It does? Ok, increment the count of pink +1.

What I find is that many of my users use mangled forms of the words in the pipal checkers. So instead of "pink" there may be "p!nk", "p1nk", "pInk", etc. From what I'm seeing pipal doesn't have mangle rules that can be applied to each string it is looking for. Since this would be something that could be used across all checkers and not just within a certain one, wanted to see what @digininja thought about adding something to the core pipal.rb script to allow for --mangle rules to be applied to strings searched for.

[digininja]

I nearly answered then realised I don't quite understand what you want. Taking the basic word checker that for:

pink pink p1nk

would come out with:

pink 2 p1nk 1

what would you want it to come out with?

[WebBreacher]

Ah. so for the string "pink", if the --mangle flag was passed, then anything mangled "p1nk", "Pink"... would be noted under the non-mangled pink string with an identifier showing that more hits were found due to the mangling.

pink*

[digininja]

so if the word list is based off the word size and contains

5ize siz3 s1z3 51z3

how would this end up in the output? I can see what you are trying to get at but I think implementing it isn't going to be that simple.

You are almost after an unmangle function to take things back to a human word but that would fail with things like Password1 where the unmangle would end up with Passwordi in the wordlist.

[WebBreacher]

Yes. A password unmangle. And you are correct that there will be FPs. Right now there are false negatives because I have "L0ve" and "lov3" in my cracked passwords and they are not being detected.

As for FPs, we already have this. For instance, I have a password of "Fredissupawesome!". The current colour_checker should find the word "red" in there but semantically, that is not the word "red" but instead "fred".

We teach our users to swap case and insert numbers for letters...I think having some kind of --demangle flag would help. Yes there will be FPs but there are FPs and FNs in the current analysis anyway.

[digininja]

what unmangles do you want?

Another though of problems, what about years or numbers on the end? If I unmnamgle winter2015 to winter20is then it wouldn't be detected by the year checker.

I would have to pass the normal and unmangled to each checker which would end up with duplicates as passing

BLUE10

through the unmangle could give

blue10 - lower case blueio - unleet + lower BLUEIO - unleet

when all three are passed into the colour checker I would get three hits for blue.

[WebBreacher]

Hmm. Aren't there already duplicates? For US zip codes "Micah'sZip21953" could already be flagged for the year 1953. So I don't think the duplicate entries is a new thing. I just accepted that there was overlap.

Another for instance, I'm writing a check (will do a pull later) for explicit words. "ass" is one but if I use the string "ass" then every time someone uses the word "password" the checker will get a FP.

I think with the --unmangle flag, each checker could be run twice. One output for the normal results and one right below (next to?) it with the unmangled.

Ultimately, pipal is trying to make it easier for analysts to examine huge lists and some of it will be spot on and some will be FPs.

[digininja]

I can add in the functions to do all this then if you want to update a checker to handle it in the way you'd expect we can see how it works.

I can see the feature being useful just hard to get right in the first place.

[WebBreacher]

I agree Robin. Password mangling is almost a one-way function...or at least requires an analyst to give eyes on it.

In my crack-file I have the following entries that are easy (for us humans) to see they are mangled words:

P@ssw0rd123password P@ssw0rd123password1 P@ssw0rd123password11 P@ssw0rd2345 P@ssw0rd33P@ssw0rd33

[WebBreacher]

An alternative to having pipal do the unmangle would be for the checker-writer to just add extra strings that were mangled to their check list. Right?

So in my file for emotions, I might have: love l0ve l0v3

It'll make the checkers longer but may cut down on the FPs. Thoughts?

[digininja]

As an aside, check out the talk on generating word lists from the Passwords Con that just finished, some really interesting info on handling word lists.

I'll add the extra feature later on.

On 10 December 2015 at 16:20, WebBreacher notifications@github.com wrote:

I agree Robin. Password mangling is almost a one-way function...or at least requires an analyst to give eyes on it.

In my crack-file I have the following entries that are easy (for us humans) to see they are mangled words:

P@ssw0rd123password P@ssw0rd123password1 P@ssw0rd123password11 P@ssw0rd2345 P@ssw0rd33P@ssw0rd33

— Reply to this email directly or view it on GitHub https://github.com/digininja/pipal/issues/23#issuecomment-163675660.

[digininja]

Lets try the unmangle before passing in first and see how that works but this could be a better option even though it is much less efficient (I think)

On 10 December 2015 at 16:22, WebBreacher notifications@github.com wrote:

An alternative to having pipal do the unmangle would be for the checker-writer to just add extra strings that were mangled to their check list. Right?

So in my file for emotions, I might have: love l0ve l0v3

It'll make the checkers longer but may cut down on the FPs. Thoughts?

— Reply to this email directly or view it on GitHub https://github.com/digininja/pipal/issues/23#issuecomment-163676151.

[WebBreacher]

I agree it is less efficient but, may have higher success rate (or lower FP rate).

Pipal does case insensitive matching right? So we wouldn't have to worry about case, just number and symbol substitution.

[digininja]

I've got a demo working but spent the last hour fighting github to get the branching working and it checked in. Will talk to a friend tomorrow who knows how to do it.

[digininja]

I beat git and got the files checked in then Windows 10 got revenge by shutting my machine down to do an update.

Check the unmangle branch.

I've got an unmangle class that does the unmangling, that is applied to the word and passed into the checkers. The unmangle_demo checker takes the original and unmangled word and does some counting on them. It doesn't work well yet as it currently depends on the order of the words in the input file but leaving it at this to see if it is what you were looking for, if it is then we can take it forward.