Autofill OTP forms

[J-Jamet]

The autofill of the OTP fields would be perfect but requires a lot of work in the realization of the parser because each form is different.

You can leave a comment in order to define the forms to study in priority.

[matesurano]

Continuation of the closed thread discussion>>>as mentioned by you in the closed thread "The notification is displayed when you select an entry in the database. This is the normal workflow for copying data. Notifications cannot be linked directly to the autofill." I understand that auto filling of TOTP is not yet implemented in Keepass DX, but after selecting the entry in the database, i am not getting any kind of notification. Really confusing....may be i am doing something wrong! I will have to reinstall and start from scratch( although i did it many times). Interestingly keepass2android is also showing similar behaviour( getting notifications only when opening links from within the app). Anyways, Thanks.

[J-Jamet]

I just retested the OTP token notifications, and it works well. Maybe the OTP of your entry is misconfigured otherwise opens a new bug if you have a problem at this level.

[matesurano]

I just retested the OTP token notifications, and it works well. Maybe the OTP of your entry is misconfigured otherwise opens a new bug if you have a problem at this level.

Are you opening links from browser or from inside Keepass DX? In case its browser and TOTP notifications are working for you then it must be something wrong on my side.Also, OTP is not misconfigured as its working with Dashlane,Enpass etc. Will test again from fresh. Just FYI, i am on android 9.

[matesurano]

Correct me if I am wrong, but I think it will be hard to implement autofill OTP forms (i am not at all a programer ,just a guess). Instead autocopy of OTP(saved along with the credentials)should be enough for this. As we can easily paste it in the respective fields. Also, i haven't come across any android PWM which autofills the OTP(again correct me if i am wrong).

[J-Jamet]

This is an important remark that you have just made. Indeed, at the moment I do not know if it is difficult to implement because I have not studied the OTP forms but you are surely right. Now I understand what you meant by creating a notification for OTP. I thought you were talking about the functionality already implemented, but what I didn't understand is that you wanted to use this method as a workaround to non-recognition of OTP forms. Which may be a good idea, but I prefer to do things in order, there may be other methods that have not yet been explored that may be more effective. I just don't have enough visibility at the moment.

[J-Jamet]

I just checked several things and did many tests:

• There are no easy ways to detect autofill fields, they seem different every time. I can detect it when there is the otp metadata in the name but that's it. It will be a big job to check each form.
• The alternative solution of launching a notification cannot work every time because it requires launching a service but there is no listener when you click on an element of the autofill. So I can only start the notification if there is only one possible value before filling the field with addDataset.

So for the moment I don't have a solution that satisfies me fully to fill in the OTP. If you absolutely want an autofill for this data, we need to study the source code of each OTP field with its metadata and do it on a case-by-case basis.

[jeslinmx]

+1 to what @matesurano said; Bitwarden on Android does the same thing (after autofilling, the OTP is automatically copied into the clipboard). Should not be too much of a security concern, since the OTP is meant to be temporary and non-reversible, but maybe for the more security-conscious an option can be included to disable this.

[mirsella]

+1 to what @matesurano said; Bitwarden on Android does the same thing (after autofilling, the OTP is automatically copied into the clipboard). Should not be too much of a security concern, since the OTP is meant to be temporary and non-reversible, but maybe for the more security-conscious an option can be included to disable this.

it's probably the best option for now. especially since this is better than nothing.

[J-Jamet]

Guys, if another app is using a method doesn't mean it's easily applicable here. It's like you completely ignore what I'm saying, putting +1 does not solve technical constraints:

The alternative solution of launching a notification cannot work every time because it requires launching a service but there is no listener when you click on an element of the autofill. So I can only start the notification if there is only one possible value before filling the field with addDataset.

I can apply the notification solution only if there is only one entry offered by the autofill. If there is more than one, the program cannot know which notification and element to copy. What are your ideas on this problem?

[J-Jamet]

I just saw that the method https://developer.android.com/reference/android/widget/RemoteViews#setOnClickPendingIntent(int,%20android.app.PendingIntent) exists. I think I can use it to identify the entry and directly copy the TOTP by fetching the response from the service. But I have to test to see if there are any issues with it.

[mirsella]

how I'm seeing it personally, is just when you select an entry from the keepassdx pop up, if there is a OTP associated with this entry it just create notification to copy the OTP.

another option would be to just copy the OTP to clipboard when auto filling a entry on a site. still manual paste in the input box.

both of these method shouldn't be too hard to implement no ? when auto filling a entry and if the option is enable, copy the OTP to clipboard or create a notification which copy it.

maybe i didn't understood some things ?

[J-Jamet]

both of these method shouldn't be too hard to implement no ? when auto filling a entry and if the option is enable, copy the OTP to clipboard or create a notification which copy it.

These are assumptions that users make without having studied the subject. It's not at all as easy as you might think. I'm just sending RemoteViews in the device's autofill service response. But since it's not KeePassDX's role to display this popup views and copy the text to the fields, I don't have a direct listener to pick up the click event either. So the only solution I see to manage a clicked element is to go through a broadcast system with a recovery of the view id by service (the method indicated above). But I don't know if it's really doable.

[mirsella]

thanks for the explanation, didn't understood you couldn't add a hook or something when the user selected a entry from the pop up. so yeah as you said you can just give all the OTP for the entry requested by the device autofill service.

[J-Jamet]

I may not have been very specific either, I am aware that the constraint is not understandable from the start. I just tested with the method described above and it seems to partially work. No need to use a broadcast, calling it from a service with the PendingIntent seems OK. But now it doesn't fill autofill anymore, so it seems to override the basic fill method. I keep you informed if I manage to have both behavior at the same time (If it's possible).

[J-Jamet]

Despite my attempts, I cannot manage to have both behaviors simultaneously if there are several entries offered by the autofill. The setOnClickPendingIntent() method breaks the autofill if used in another way.

Finally, if the OTP and the password are in 2 different databases, you just have to use another way to fill in the OTP token when you are on the second database:

• with the magikeyboard
• or with the clipboard notification

So from a functional point of view, it's not a big deal. I remind you that there is no interest in having a TOTP in the same database as your password for the same entry, this second factor is useless in this case.

I haven't looked at how bitwarden works on this point, but there must be two different connections to access OTPs or additional encryption for this item. As it is necessary to make an additional database with KeePass, the problem normally no longer arises.

I can also improve the manual copy of the OTP token to make the second database workflow easier to use. Tell me if you have any ideas on this.

[J-Jamet]

Otherwise I return to the primary goal of detecting TOTP fields. HTML attribute autocomplete has the one-time-code value, if it is used more and more, it will be more and more recognized. And we will add non-standardized formats on a case-by-case basis.

[cmd1982]

You can leave a comment in order to define the forms to study in priority.

I wish I could fill in the paypal totp. I think you need the otpCode value here? Sorry I'm not a programmer just want to help.

<input type="tel" name="otpCode" value="" class="hasHelp hasError" placeholder="Beveiligingscode" autocomplete="off" data-nemo="twofactorOTPInput" maxlength="6" id="otpCode" aria-describedby="otpCode">

And here some other codes from other websites.

<input class="ahoy-input ahoy-input-large ahoy-margin-top-2" id="mfa_token_input" autocomplete="off" type="text" placeholder="MFA token" onkeypress="return submitenter(this,event)" name="mfa_token">

<input _ngcontent-c1="" class="form-control hide-placeholder-on-desktop-view ng-pristine ng-invalid ng-touched" formcontrolname="twoFactorCodeInput" id="twoFactorCode" type="text" placeholder="Verificatiecode">

microsoftonline.com <input id="idTxtBx_SAOTCC_OTC" name="otc" class="form-control" type="tel" autocomplete="off" aria-required="true" data-bind=" attr: { 'maxlength': otcLength, 'aria-labelledby': 'idTxtBx_SAOTCC_OTC idDiv_SAOTCC_Title idDiv_SAOTCC_Description', 'aria-describedby':'idSpan_SAOTCC_Error_OTC' }, css: { 'has-error': error }, textInput: otcInputTextbox.value, ariaLabel: str['CT_SAOTCC_STR_OTC_TBHint'], hasFocusEx: otcInputTextbox.focused, placeholder: $placeholderText" maxlength="6" aria-labelledby="idTxtBx_SAOTCC_OTC idDiv_SAOTCC_Title idDiv_SAOTCC_Description" aria-describedby="idSpan_SAOTCC_Error_OTC" aria-label="Code" placeholder="Code">

Looks like everyone is using an other value. Is it the name= value you need? I can make a list if you want? I have like 24 websites with TOTP and this is for me a must have feature.

[J-Jamet]

Thank you @cmd1982. This is the type of information needed to make the parser. Do not hesitate to put the site address associated with the form code. Here the name and type are important, but if we find other elements that allow us to specifically target the field as an otp type, that's good too. The goal is to generalize the rules as much as possible without making false positives.

[gabeweb]

It would also be valid to copy the contents of the generated OTP key (in addition to the option to display an icon to populate the OTP key field). Enpass also does this on PC and mobile and is very helpful under certain conditions (sites that do not allow you to fill in automatically or sites that do not allow copying/pasting content).

[famewolf]

I'd be happy with auto copy to clipboard of totp if a single entry match. m1finance.com has a checkbox so you can enter user/pass/otp all on same screen or if you just enter user/pass it then prompts for otp on 2nd screen. Would any of the code from keepassxc be relevant because they fill otp at click of button on the desktop after entering user and pass. I was hoping to migrate to keepass specifically because I'm tired of swapping windows to launch aegis, logging in, copy otp, swap back to app, paste.

[xf-]

@J-Jamet maybe take look at the keepassxc browser plugin (firefox & chromium) and topt part. Yeah it is a different language, but the detection works on most pages. https://github.com/keepassxreboot/keepassxc-browser/blob/develop/keepassxc-browser/content/totp-field.js

[J-Jamet]

Thank you @xf- , this will indeed be a good entry point for the creation of the feature! :)

[J-Jamet]

Ok guys, it's very complicated to test code blindly without having a form to test with, so if you have the ability to create html files that contain OTP forms from your favorite services it would be much easier for debugging.

You can generate a file from the source code of the page from your phone. (on chrome based browser add view-source: before the URL address then copy paste the code source in an html file and remove your login credentials from the code if there are any traces) Thank you for your help

[famewolf]

Here is one from m1finance but not sure it's helpful.

On August 23, 2021 9:48:19 AM Jérémy JAMET ***@***.***> wrote: > > Ok guys, it's very complicated to test code blindly without having a form > to test with, so if you have the ability to create html files that contain > OTP forms from your favorite services it would be much easier for debugging. > You can generate a file from the source code of the page from your phone. > (on chrome based browser add view-source: before the URL address then copy > paste the code source in an html file and remove your login credentials > from the code if there are any traces) > Thank you for your help— > You are receiving this because you commented. > Reply to this email directly, view it on GitHub, or unsubscribe. > Triage notifications on the go with GitHub Mobile for iOS or Android.

[J-Jamet]

@famewolf Unfortunately that does not help, it would take the structure of the DOM in fact it would be better. And be careful because your copy paste contains id info.

[xf-]

@J-Jamet I think starting with most common stuff like GitHub would be a good starting point. Some sites have some extra steps if a second 2FA is registered as well. For example Nextcloud using an OTP or hardware Token. I think Google has the same option.

[famewolf]

@famewolf Unfortunately that does not help, it would take the structure of the DOM in fact it would be better. And be careful because your copy paste contains id info.

Hmm I tried looking through it but obviously didn't know what to look for. Do I need to change my password on that site?

[mirsella]

Hey i hope these helps : github and nextcloud

[J-Jamet]

As this is a difficult feature, I will already release a version 3.0.0_beta_01 without it. Then I will improve the OTP code for the next beta. It will allow to have a feedback of the bugs with a shorter cycle.

[J-Jamet]

OK guys. Bad news, I made few tests and the OTP forms are not even parsed by the autofill. So I'm out of solutions.

• Native Autofill OTP form : Does not fit in the parser, so I can't recognize the form metadata. Maybe the structure of the form is not properly constructed, I don't really know why. (tested with @mirsella forms on a local server, note that the main github credentials form works on the local server)

• Compatibility mode Autofill OTP form : Recognizes only the fields as text, without metadata so not usable. (Cannot be differentiated from a username field)

• TOTP in the clipboard during Autofill : Can't be done natively because the views of the autofill elements are delegated to the system and there is no listener for an autofill selection event (see https://github.com/Kunzisoft/KeePassDX/issues/553#issuecomment-745424894). A hack that allows copying before displaying the autofill (as Bitwarden does) is possible but has huge cons :

  • There is no point in displaying autofill if the second factor is separate from the main identifiers (which must be the case, otherwise why implementing 2FA?). The autofill will fill empty fields in main credential form.
  • The copy that is made cannot be done on all the fields returned by the autofill, so in the best case, only the first OTP token will be copied in the clipboard if several entries are retrieved during a domain search.
  • The time between the copy of the token in the clipboard which is done at the display of the autofill and the validation of the form leaves time for the token to expire.

So the only thing I can suggest is to make it easier to copy a token from the list of entries. Now the token (and its expiration time) is displayed in the list without opening the details of an entry, I can add the copy in the clipboard there. What do you think about it?

Note : If someone can recognize natively OTP forms from Android autofill, feel free to propose your solution.

[uduerholz]

I also had a look into this and finished a first implementation. My approach is:

Don't try to parse and guess the correct form field for the OTP (with one exception: the website uses the correct autocomplete attribute "one-time-code" as specified here) Instead, add the OTP always to the form field that has the current focus if and only if there exists a database entry that is already associated with the website or android app and has an OTP secret.

I did some tests and it works e.g. with the current github webpage (and chrome browser).

@J-Jamet I can create a pull request, if you have time to look at it.

[J-Jamet]

@uduerholz Yes I can look of course. I'm afraid that it fills the fields too often but it can be a good approach in practice so I'm curious about the test results! :) (And thank you for your involvement)

[uduerholz]

I was not clear about this: The field with the focus is not filled automatically, there is an extra item in the list (or extra inline suggestion) to select the OTP (and only the OTP) for the field that triggered the autofill.

[uduerholz]

Hm, this also means that you will see an autofill view popup every time you fill out a form on a website where you login with OTP. Probably better to prevent this.

[J-Jamet]

As you say, your solution is too generic. (https://github.com/Kunzisoft/KeePassDX/pull/1079#issuecomment-913756807)

[uduerholz]

Yes, I realized this problem too late. Somehow all solutions have their own drawbacks. It would be much easier if web developers would use the correct "hints". For now I would suggest to make it easier to copy the OTP to the clipboard, as you said.

[J-Jamet]

One of the solutions would be to create an algorithm with a pseudo language to describe the fields of the forms to be filled, it would be very long because obliged to indicate a configuration for each OTP form but I do not see how to do otherwise. Linked to https://github.com/Kunzisoft/KeePassDX/issues/1287

[famewolf]

I thought this had been mentioned before but why can't we use the same implementation keepassxc's browser plugin uses? It's also open source and correctly populates totp where appropriate or at a minimum gives a button that lets YOU decide whether to put totp in the field.

[J-Jamet]

The constraints of Android devices are not the same, we can not use the same recognition system for applications. But for website forms it should work, then we have to do the conversions and manage the architecture which is different and Android will surely not let you select easily a form field with special permission. If you are motivated, YOU can implement the feature and make a pull request! :)

[famewolf]

If you wanted it written in mainframe cobol and I wasn't medically retired I might consider it. As is I'll merely pose the question. Firefox fully supports plugins on android. Other plugins like ublock, decentraleye's, dark reader etc have been ported. A similar plugin might integrate with keepassxc to allow some similar detection method. Merely a thought.

[J-Jamet]

Yes, of course it's a good idea, but the tricky part is to manage the porting of the detection code. We have to study the feasibility while adapting the constraints. I work when I can on this project and I have to prioritize the features especially since this one will take a lot of time and there are already implemented methods for form filling. I will study after the 3.5.0 version.

[WhyNotHugo]

I'm also very much looking for this feature. Given that it seems to be hard, I'm wondering what your workarounds are right now. In its absence, currently, my login flow is:

• On the login website, pick the KeePassDX entry.
• Open KeePassDX.
• Search for the same entry I've just used. (1)
• Open the entry. (2)
• Click to copy the OTP
• Switch back to the original site.
• Paste the OTP.

These are a lot of extra steps for the OTP!

A note on the above (1): this search is a bit silly. KPDX has just shown me an autofill-dialog with this entry, and the entry has an OTP. At this point, it's sane to assume that I'm coming for this entry's OTP, so maybe suggest it at at this point? Implementing this implies keeping around recently-used items, which would definitely be useful for whatever is the ultimate fix for this ticket.

On the above (2): While proofreading this, I realised I can click on the OTP directly to copy it. Maybe showing a clipboard icon would make this super-obvious to newcomers like myself?

Do these two improvements sound reasonable? Are they in-scope here, or should I rather open new issues for them?

Extra note: The "copy to clipboard on autofill" is also done by 1Password, I don't think it's unfeasible.

[J-Jamet]

At this point, it's sane to assume that I'm coming for this entry's OTP, so maybe suggest it at at this point?

At this point, the use of the Autofill is over. After that it is a search like any other. The autofill workflow requires the detection of the form that contains the OTP, but this is not standardized, and difficult to set up (linked to #1272 #1287). On top of that, It's not obvious, but using an OTP in the same database is not recommended because there is no point in having a password and an OTP in the same vault so I'm not going to promote the behavior.

Maybe showing a clipboard icon would make this super-obvious to newcomers like myself?

Maybe will overload the user interface but I will look at how to improve it.

Extra note: The "copy to clipboard on autofill" is also done by 1Password, I don't think it's unfeasible.

It's not a question of not feasible, it's a bad workaround. Ref section TOTP in the clipboard during Autofill : in my previous comment : https://github.com/Kunzisoft/KeePassDX/issues/553#issuecomment-912936390

The clean solution is to recognize each OTP form for Autofill, but for that you have to create a specific parser for each form. https://github.com/Kunzisoft/KeePassDX/issues/1287

[WhyNotHugo]

At this point, the use of the Autofill is over. After that it is a search like any other. The autofill workflow requires the detection of the form that contains the OTP, but this is not standardized, and difficult to set up (linked to https://github.com/Kunzisoft/KeePassDX/issues/1272 https://github.com/Kunzisoft/KeePassDX/issues/1287).

So after the user taps the auto-fill choice, KeePassDX does not receive any event/message/read? The passwords have already been sent too?

On top of that, It's not obvious, but using an OTP in the same database is not recommended because there is no point in having a password and an OTP in the same vault so I'm not going to promote the behavior.

I'm aware that it's terrible behaviour, but some site don't support hardware 2fa, nor anything sensible. A second vault is a PITA since I need to go through the whole unlocking process twice.

FWIW, using a second vault would also be pointless, or even a different app on the same device. A phone is still.

Maybe will overload the user interface but I will look at how to improve it.

True. Maybe putting a clipboard icon and an eye icon. The eye one showing the OTP code?

It's not a question of not feasible, it's a bad workaround. Ref section TOTP in the clipboard during Autofill :

Regarding these:

There is no point in displaying autofill if the second factor is separate from the main identifiers (which must be the case, otherwise why implementing 2FA?).

Because the darn website doesn't support well hardware 2fa, I don't carry a second phone, and they data there is not worth the price of a hardware TOTP hardware device.

The time between the copy of the token in the clipboard which is done at the display of the autofill and the validation of the form leaves time for the token to expire.

This should be fine assuming the next page loads quickly. The same caveat also exists in copying from one application and pasting in another, there's a possible delay. Websites usually give some leeway to prevent this from being an issue anyway, but implementing auto-copy-to-clipboard wouldn't make the risk of this expiration happening any worse.

The clean solution is to recognize each OTP form for Autofill, but for that you have to create a specific parser for each form. https://github.com/Kunzisoft/KeePassDX/issues/1287

How do other password managers implement this?

[J-Jamet]

So after the user taps the auto-fill choice, KeePassDX does not receive any event/message/read? The passwords have already been sent too?

No : https://github.com/Kunzisoft/KeePassDX/issues/553#issuecomment-745245875

I'm aware that it's terrible behaviour, but some site don't support hardware 2fa, nor anything sensible. A second vault is a PITA since I need to go through the whole unlocking process twice. Because the darn website doesn't support well hardware 2fa, I don't carry a second phone, and they data there is not worth the price of a hardware TOTP hardware device.

Two processes -> two factors: very strange. :joy:

This is normal for me. If the other managers take shortcuts, good for them, but in this case they might as well put 5 factors to give the illusion of security and put all the unlocking system in the same process, but I don't see the point, except to make it more complex during usage. It does not bring more security.

FWIW, using a second vault would also be pointless, or even a different app on the same device. A phone is still.

I strongly disagree, the purpose of KeePass is to provide a secure environment for storing sensitive data.

A hardware key allows to separate the sensitive containers, if the goal is to do it in a software way with the TOTP, it is also a very good practice to separate the sensitive containers.

So it doesn't prevent to use a separate database or another application for OTP token generation. I'm aware that having to manage two open databases at the same time is not done yet but it is planned.

If the databases are properly encrypted, a stolen phone will not be able to provide sensitive information easily. Also, if a malicious person has gained access to a database, it will still be blocked. Hence the two-factor principle. Of course it's better to have a hardware key, but a second database provides a second authentication solution.

How do other password managers implement this?

They have not done anything on this feature since they use the workaround, it is to create.

[gabeweb]

Hi,

When I need to loggin in a website that requires 2FA, I just select the Magikeyboard before it. When the website doesn't requires 2FA, just using the autofill.

The other password manager that is some "friendly" with 2FA (in mobile and desktop) is Enpass, because you can activate the option to copy automatically (and for a few seconds) the 2FA code from clipboard.

[J-Jamet]

Thanks @gabeweb for the tip with the Magikeyboard.

I am aware that the automatic filling of the OTP is not done and that it is a definite lack. But I'm not going to workaround and take shortcuts just because others are doing it. This is a global problem because every form is different and nothing has been standardized. So might as well work directly on standardizing OTP form recognition.

[WhyNotHugo]

So might as well work directly on standardizing OTP form recognition.

Is it somehow possible to detect that the current page loaded in the same tab / session as the last autofill?

This, and the presence of configured TOTP for an entry should be a strong hint that there's a TOTP field on screen.

[J-Jamet]

Is it somehow possible to detect that the current page loaded in the same tab / session as the last autofill?

Unfortunately not, autofill works as a system-independent service, it provides the application with restricted access to a metadata representation of the elements of a page. The application can just choose the metadata that best matches the form fields to be filled in, and the representation of a view is sent to this service that will take care of displaying it and managing the filling.

The problem with the OTP form is that it is not possible to detect generically which metadata corresponds to these fields. So the only solution is to make a representation of a page in advance to make the link between the recovered metadata and a real OTP field. To do this we need to create a pseudo language so that we can represent the structure to be recognized for each URL.

It is the creation of the strong hint that concretely has to be done.

[gabeweb]

I think that the only "friendly" way that KeePassDX could offer for the filling of 2FA codes is that in the upper banner that is displayed when the vault is open, a button appears that copies the 2FA code of the website in question. If I remember correctly, this and other buttons related to the vault entry appear when you first open KeePassDX and then try to open the website or app to log into (which is more steps to complete).

To do this, KeePassDX should previously have the clipboard permission or access option activated, along with a stipulated time to keep the copied code.

The entire process of copying the code would be manual and operated only by the user, not automated as in the case of Enpass (however, for Enpass to copy the 2FA code, the option must first be activated).