More detailed password character requirements?

[atrigent]

I have just come across a website which balks at some of the special characters that PFP uses in its passwords. Specifically, this is the message I get:

Ensure your new password does not contain the following characters ( ) " ' , ; < >

This website's password requirements specify that the password should have at least one special character, so I cannot fully disable special characters for this password. I see in the source code that PFP already excludes quotes, but have you considered allowing the exclusion of other characters for specific passwords?

[palant]

That's a first one. Could you provide a link to this website?

[ghost]

This was already proposed in https://github.com/palant/pfp/issues/72 but without example.

[atrigent]

Here's the sign-up page: https://www.olt.com/main/home/getstarted.asp, it's an online US tax filing website. These kind if restrictions always make me worried that the website isn't confident in the measures they've taken to avoid SQL injections, but... such is life, I guess.

[palant]

I went through their list and the following PfP special characters aren't allowed there: (),:;<>. This isn't about SQL injections, completely arbitrary and bogus.

[palant]

The proposal in #72 isn't really a good way to solve this. If a website has too specific requirements on special characters, we shouldn't expect people to compare two lists to figure out which characters cause issues. The better solution should allow defining a "custom character group" where people could just paste the "allowed special characters" list from the website. Creating a usable yet not too intrusive user interface for this will be challenging however.

[ghost]

In my experience sites often don't specify list of allowed characters and most users don't know what characters are allowed in pfp itself. Doing trial and error with blacklist would be easier to do in such case than creating whitelist.

[palant]

Sites that simply blacklist some characters typically don't require special characters to be present - that's the easy scenario, we already handle that. When special characters are required, a list is usually given.

[charliemb2]

I recommend two or three check marks for special characters. A first checkmark enables only - and _. The second adds only typeable characters (characters which are visible on a standard keyboard or a subset for all country keyboards) except omit all characters used in command line (CLI) processing in Windows or Linux or MacOS, and by that I mean for windows omitting | < > $ \ / " ', and for Linux at least Bash equivalents, etc.. Then a third checkbox allows all characters including non-typeable characters. Of course omit spaces, they are not only problematic but also many sites don't allow spaces.

[jotaf98]

I realize someone may have encountered sites with different requirements, but all sites I've seen so far that demand using special characters are perfectly happy when you just add underscores (_), which in a way can be seen as the "least controversial" special character (i.e. usable in variable names in all languages, and file names in all OS's).

So maybe just adding that option on the side would be enough to appease these sites. In any case it's just a suggestion.

[palant]

I think that the best way to solve this is to add a fifth checkmark labeled "custom" when selecting allowed characters. It should be unchecked by default, when checked an input field would appear to let you specify the characters. Whitespace and duplicate characters should be removed automatically, so you could simply copy the list from the site's error message - uncheck the usual special characters and you are done.

This would of course not be the recommended solution. There is a significant drawback: recovering passwords would be considerably more complicated. So we might want to deemphasize this somehow.

[charliemb2]

Brilliant. That should work, assuming of course that this is implemented per site with a potentially different list of characters per site.

To make recovery easier, one could attempt to always use the same subset. Inconvenience would then only occur for the exceptions. -- Charlie

On June 6, 2019 5:30:23 AM EDT, Wladimir Palant notifications@github.com wrote:

I think that the best way to solve this is to add a fifth checkmark labeled "custom" when selecting allowed characters. It should be unchecked by default, when checked an input field would appear to let you specify the characters. Whitespace and duplicate characters should be removed automatically, so you could simply copy the list from the site's error message - uncheck the usual special characters and you are done.

This would of course not be the recommended solution. There is a significant drawback: recovering passwords would be considerably more complicated. So we might want to deemphasize this somehow.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/palant/pfp/issues/88#issuecomment-499421342

[palant]

It will be implemented on per-password basis, and automatically taken over when you generate a new revision of a password or something like this. We don't keep this kind of state on per-site basis, and globally defined character sets don't make much sense given how random the requirements of some sites are.

[charliemb2]

Ok. Thanks. -- Charlie

On June 6, 2019 9:15:07 AM EDT, Wladimir Palant notifications@github.com wrote:

It will be implemented on per-password basis, and automatically taken over when you generate a new revision of a password or something like this. We don't keep this kind of state on per-site basis, and globally defined character sets don't make much sense given how random the requirements of some sites are.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/palant/pfp/issues/88#issuecomment-499488529

[adrium]

I have been using my fork of the extension since mid 2018 and migrated most passwords to generated ones. The database contains around 120 accounts with generated passwords now. I wanted to share some experience here.

TLDR

1. As users, how do you handle generated passwords that are not accepted?
2. What about a new default algorithm?

Background

I am using 3 "profiles" for the settings:

• Normal: Default settings
• Simple: Length 8, no symbols (passwords on phones like banking apps)
• PIN: 4 or 6 digits

There are 4 exceptions where this does not work:

• ebay.com
• shop.m-budget.migros.ch
• candidates.cambridgeenglish.org
• corner.ch

eBay is a very well known site. Their requirements are as follows (as of today):

• Use 6 to 64 characters.
• Besides letters, include at least a number or symbol !@#$%^*-_+=&~ .
• Password is case sensitive.

When I changed my password a few months ago, they did not list the character set and since they write number or symbol, I simply removed all symbols entirely. This is where I discovered it first that some symbols do not seem work.

The shop.m-budget.migros.ch is an interesting story. I just migrated those passwords today and this experience prompted me to write here. The page reported that the password was changed successfully, but when I tried to log in, neither the old, nor the new password worked! I suspected that the password again contained an unacceptable character. The actual password (letters and numbers replaced) was: aaa]&_?bbbbbb-<c I tried to login again using aaa]&_?bbbbbb- and it worked! That is creepy!

The latter two pages have the same requirements: at least 8 characters, containing at least 2 numbers Unfortunately, the default algorithm produced only one number.

User Experiences

As users, how do you handle those situations? In the latter two, I just appended a 1 to the password. In the former two, it was actually possible to just use a password without symbols.

Anyway, the unpleasant consequence is that more state needs to be tracked for those logins.

Different Algorithm

Since there are still sites that do not work with arbitrary passwords, how about changing the default generation algorithm? All the described problems could be solved by using equal probabilities for all character sets and by restricting the symbols to a small subset. Sure, it decreases security, but it could be compensated by generating longer passwords by default.

[jotaf98]

Since there will always be some corner case which asks for even more state, what about going a different route:

A community-maintained list of domains and associated sets, versioned (and with a "backwards compatible at all costs" rule, only changing the set / bumping the version when the site actually changes the acceptable set)?

[adrium]

Interesting approach... This leads to another difficulty though: If the list is not included in the extension, it would have to be loaded online. Which means that the list should be authenticated.

What about the problem of "containing at least 2 digits"?

[Maryse47]

You may try to increase length of password which would get a chance for additional or number special character. You may also generate random password and store it in extension. Weakening algorithm would undermine the purpose of this project.

[adrium]

@Maryse47 Please note that weakening the algorithm has never been proposed.

[jotaf98]

Yes, this would keep the same algorithm but "eliminate state", by making all state a shared feature of all users (the community-maintained list of password requirements per-site).

As it stands, carrying state around, especially for sites with strange rules, is messy and can easily make a password impossible to recover ("did I choose 8 or 9 characters for this site? which special characters?" -- now there would only be one choice).

As for security, the worst that a compromised online list would achieve would be to not let you use your passwords temporarily (until you revert to the previous safe version). It would not have a chance of leaking any information.

[Maryse47]

Please note that weakening the algorithm has never been proposed.

@adrium I thought you literally proposed this:

Different Algorithm

Since there are still sites that do not work with arbitrary passwords, how about changing the default generation algorithm? All the described problems could be solved by using equal probabilities for all character sets and by restricting the symbols to a small subset. Sure, it decreases security, but it could be compensated by generating longer passwords by default.

[adrium]

@jotaf98 The approach of "eliminating state" is interesting. I am not fully convinced though... It makes things more complex and it can not be fully used offline anymore. It may be a matter of taste in the end, but I would rather go for a "dumb-site safe algorithm". Also, a compromised list is a problem, if you use it while generating new passwords.

How would you solve the 2 digits problem? Increasing the password length can not guarantee this because it might differ for different master passwords. This can only be solved by another algorithm.

@Maryse47:

Sure, it decreases security, but it could be compensated by generating longer passwords by default.

By the way, there are already built-in security/compatibility tradeoffs (lib/crypto.js):

I, l, O, 0, 1 excluded because of potential confusion. ", ', \ excluded because of common bugs in web interfaces (magic quotes).

The second is understandable, but I have no idea what the first is about - I have never seen any of those passwords.

I made a quick comparison using KeePass' entropy estimation. The master password is secret and the site is example.com

Original Algorithm

Username  Password            Bits
ttxshetl  d{ujwb2ag#a(myxP    102
pejeuegq  >H,z/K7,Jpbku8S)    99
bkvidqoj  &xckZ!}5{bHF-fEs    102
dosktgyf  C7MSGkTCdzV??6H4    86
cqvvkvgt  {b.;G{t;zuJ@F<U6    96
hvedapip  5.K|(Z>CvhwGW)mc    106
auzyjgaf  {k/R6(|TEnpg3Hun    102
gnpxdmfp  zSh62g/G2.KmN6~T    96
mmymflfx  R.fkSC2F_=c^#b49    102
jtgarbfj  Fw6e|J)QGP.ecSj5    99

Reduced symbol charset to !@#%*-_+

Username  Password            Bits
ttxshetl  d#%jwy2xDMUR6+UW    91
pejeuegq  k6baeKGykpyhRmt*    95
bkvidqoj  tUzhd!v5#bH4zffs    88
dosktgyf  C7MtGG-dXaVmHj6D    95
cqvvkvgt  #VXg5t#2aR77g3v*    88
hvedapip  5AmRRZkCSE-GW_Ha    96
auzyjgaf  tGBRF-%TE782fj%n    98
gnpxdmfp  WSfFBgBGeAKH!FT3    90
mmymflfx  %dZktCBgsFz!Mb49    101
jtgarbfj  F-6cuJSQG@XeatgE    102

Reduced symbol charset and length 18

Username  Password            Bits
ttxshetl  d#%jwy2xDMUR6+UW-J  103
pejeuegq  k6baeKGykpyhRmt_Eb  101
bkvidqoj  tUzhd!v5#bH4zffsr6  101
dosktgyf  C7MtGG-dXaVmHj6DUj  104
cqvvkvgt  #VXg5t#2aR77g3v*GS  101
hvedapip  5AmRRZkCSE-GW_HaX4  109
auzyjgaf  tGBRF-%TE782fj%nD-  107
gnpxdmfp  WSfFBgBGeAKH!FT-k4  99
mmymflfx  %dZktCBgsFz!Mb49gb  107
jtgarbfj  F-6cuJSQG@XeatgECe  109

Full charset

Username  Password            Bits
ttxshetl  d2^it@U?_F)J=|ik    99
pejeuegq  *s"}%GBZhm@&fOo0    106
bkvidqoj  Ui[&H2=X2bEq0fdp    102
dosktgyf  zZHoD|Bb,}P+8Msy    106
cqvvkvgt  2*P'r;]S}ftXeTq4    106
hvedapip  X1i&JT*zg``DQ8}8    99
auzyjgaf  ;|2LA73NB>?/Jg^l    99
gnpxdmfp  kM$Awg2DI1G}wA(B    96
mmymflfx  z$.jozwe:6[0FbW1    99
jtgarbfj  C`Y!<FKKDxPe8o%z    99

All of them are by far in the moderate category according to KeePass' assesment. Personally, I think the differences are negligible. If this is of concern, it should be questioned if using a deterministic generation algorithm is appropriate at all.

[Maryse47]

I, l, O, 0, 1 excluded because of potential confusion.

I wasn't aware of this and it's surprising (negatively). Couldn't it it just bundle some proper font? I don't think doctoring created password further is the proper way to go. Some sites (even very security sensitive) have quite low maximum char limit (below 16) so you can't rely on increasing length everywhere.

[jotaf98]

@jotaf98 The approach of "eliminating state" is interesting. I am not fully convinced though... It makes things more complex and it can not be fully used offline anymore. It may be a matter of taste in the end, but I would rather go for a "dumb-site safe algorithm". Also, a compromised list is a problem, if you use it while generating new passwords.

Well, another way to see it is: it's just a (extreme) convenience. You can set the state manually, sure. Completely offline. (Though using passwords usually requires you to be online...)

But for 99% of the cases, users want things to "just work". Fetching a well-curated list of states (say, from a GitHub repo) would achieve that. The overhead can be minimal if people send PRs with the state they set manually for sites they want to use.

How would you solve the 2 digits problem? Increasing the password length can not guarantee this because it might differ for different master passwords. This can only be solved by another algorithm.

Yes, or more state ("exact number of digits", which could revert to the current algo if -1). Adding more state like this is not bad, if it's handled automatically like we described -- popping two birds with one stone. But I do understand not wanting to add the complexity of fetching a file periodically.

[adrium]

It is now possible to select a new generation method in my fork. It enforces 2 occurrences of each charset, and uses all letters and digits, but less symbols.

All sites that have been reported in this thread work with this generation method.

There was a password policy hall of shame that collected sites with bad policies. But unfortunately, it is outdated and was not considered because of that.

For the examples I mentioned previously, symbols could be enabled again and the password quality increased by 10 bits.

[adrium]

In the past few days, I have come across two more websites that don't accept the passwords generated by pfp. Both of them contained a < and the password was accepted when it was removed.

The issue most likely is not a buggy website, but a wrongly/too sensibly configured WAF. One aspects of my job is actually maintaining complex WAF configurations and I have come across similar issues. An angle bracket is very likely to trigger a protection rule. One of the sites also produced an error that looks like coming from a WAF supporting that theory.

The "compatibility method" in my fork works very well in all cases and I am happy that it is easy to enable. Feel free to cherry pick the commits.

[palant]

As of PfP 3.0, the randomly generated password can be edited manually. This should do for the cases where the default character classes won’t work.