[$32000] Desktop - Autofill doesn't work during login or when adding a debit card

[kavimuru]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Issue was found when executing #9294

Action Performed:

1. Open app
2. Use autofill on the login screen
3. Navigate to Settings>Payments>Add Debit card
4. Use autofill

Expected Result:

Autofill should works during login Autofill should works while adding new Debit card

Actual Result:

No autofill for fields at login page No autofill for all fields while adding new Debit card

Workaround:

unknown

Platform:

Where is this issue occurring?

• Desktop App

Version Number: Version 1.1.86-1

Reproducible in staging?: y

Reproducible in production?: y

Email or phone of affected tester (no customers):

Logs: https://stackoverflow.com/c/expensify/questions/4856

Notes/Photos/Videos:

https://user-images.githubusercontent.com/43996225/181026173-67396956-4e27-4886-9d7b-047dd470e730.mp4

Upwork job URL: https://www.upwork.com/jobs/~01a9e983bc39d24141

Issue reported by: Applause internal team

Slack conversation:

View all open jobs on GitHub

[parasharrajat]

I am looking at it today for sure but it can take some time to go over all proposals.

[parasharrajat]

Half way through the proposals. This is my top priority.

[tomivs]

I'll update my proposal in a few hours.

[parasharrajat]

Ok. I have reviewed a lot of things. I will try my best to cover everything but ping me if I miss something.

First, let's resolve the Node-keytar and safestorage GD.

Node-keytar => Use system Keychain to store passwords for different usernames for a particular service. By default, the service which stores the credentials will be able to access those without a password prompt.

Can other process access these passwords?

Yes, by entering the prompted password. But it does not make it unsafe. You can change the passwords or delete them by simply going to the system keychain and feeding the user password. If the user's account password is already compromised or an attacker has access to the system. You can not guarantee any safety. (You can read the terms and conditions somewhere on any apple product. I am sure they must have denied taking responsibility for such incidents)

If the user's password is not compromised as well as the system(the user is prompted to feed the password, it is impossible for any application to programmatically feed the information in the UI prompt unless the system is compromised), there is no way a password can be stolen by any application.

Pros:

1. A system-level access. These keychains can be synced in apple(if configured correctly) cloud and give users a seamless sign-in experience. I think we may have to save data in the system keychain instead of custom service. In other words, the same apple id user will see the autofill suggestions on all of their apple devices.

Cons:

1. Incomplete solution. We won't be able to save debit card info AFAICT (let me know if I am wrong here).
2. Might be slower. greater read time.
3. Extra logic is needed to manage the other type of autofill.

safeStorage => Use System keychain process to store the encryption seed/key. It will create a new entry in the system keychain for our app. Again, which can be changed or accessed via any app if they know the user password. It does not store app credentials in the system keychain.

Pros:

Allows to easily encrypt data without the overhead of managing encryption keys.

Cons

1. it is not a go-to solution to store user passwords.
2. we will have to manage the encrypted data ourselves.
3. Change to encryption keys will break the existing data. Require logic to detect the decryption failure and take action about the stored data.
4. Extra overhead of managing the data.

I think both safeStorage and node-keytar can be combined together to give the user the best autofill experience. However, a single safeStorage can be used to create a solution to manage all types of autofill. Storing passwords in userland is just not considered good practice even if encryption is used. However, it might be safe to store the password with encryption.

We will have to think about it again. I will come back to it.

---

Note: I am not a security expert and I am not aware of any loopholes in the implementation of both of these packages. There might be bugs leading to unauthorized access. I am considering the positive side where these packages are working as intended.

[parasharrajat]

Common challenges for this issue.

1. Autofill popup implementation.
2. Autofill information storage.
3. Which process will handle autofill main, renderer, or both?
4. Challenges of installing/maintaining the library you are proposing e.g. node-keytar or safestorage?

Can all of you please answer these questions in your existing proposal at the end of it?

[parasharrajat]

@ntdiary Autofill library sounds like a complex solution. AFAICT, there will be many challenges with it. It might not work with all the forms. But yeah, we can focus on our app first and make it work.

Let us know your estimated timeline and design plan for the same if you are considering going for it to understand better.

can fill more than one field at a time, for example, it should also fill password after selecting the username. can filter candidates. can manage the passwords or cards data (add / edit / delete).

I agree with these.

Your videos are looking good too. But the proposal is lacking a lot of information. I hope you can answer my questions from the previous comment.

[parasharrajat]

@azimgd Your proposal is by far the most complete one.

It would be good if autofill works by filling all the fields in the form. for example, the user selects a username on the email field and the password gets filled in.

Questions to all:

1. Will it be beneficial to show the native popup around for autofill? or custom popup is best to manage?

---

Things need discussion:

1. Will Onyx be a good target to store the data(encryption or not)?
2. Should we show a native autofill popup or a custom?

---

Status: waiting for the updates on the existing proposals.

[tomivs]

Incomplete solution. We won't be able to save debit card info AFAICT (let me know if I am wrong here).

Well, node-keytar's API calls it "password" but you can store any string. So we could still store card information.

And regarding the things that need discussion:

1. Will Onyx be a good target to store the data(encryption or not)?

I think Onyx is good for managing and getting state throughout an app. In this case, do we need to use password or card information state throughout the app? I don't think so. The password/card information should only be there when submitting the form.

2. Should we show a native autofill popup or a custom?

I'm planning on implementing something like datalist for the suggestions. That way it would be consistent with the browser version. And maybe an icon next to the input field that would show a modal/popup to manage passwords.

[azimgd]

RE: SafeStorage Cons

it is not a go-to solution to store user passwords. we will have to manage the encrypted data ourselves. Change to encryption keys will break the existing data. Require logic to detect the decryption failure and take action about the stored data. Extra overhead of managing the data.

While I don't agree with the most points being a con, my main idea here is to have a single source of truth being Onyx. Data stored there is encrypted, and encryption key is stored in the OS Keychain.

Autofill information storage.

An alternative solution (partial proposal upd on: https://github.com/Expensify/App/issues/10107#issuecomment-1275606422) is to store encrypted credentials using JSON directly inside Keychain using node-keytar.

JSON Object may look like this:

{
  'debit-card': [
    {
      label: 'John Doe Visa **72',
      form: {
        fullname: 'John',
        number: '1234123412341234',
        month: 12,
        year: 2024,
      }
    }
  ],
  'authorization': [
    label: 'john@gmail.com',
    form: {
      username: 'john@gmail.com',
      password: '12341234',
    }
  ]
}

Autofill popup implementation.

https://user-images.githubusercontent.com/4882133/197191516-b5acb91d-292d-494c-ae40-f1de0cdbc326.mov

Here is the initial prototype for what we are trying to achieve: https://github.com/azimgd/expensify-app/pull/1/files I'm missing some pieces that I'm planning to implement once assigned:

1. Add an identifier that will be used to determine Form's autofill type based on the schema above

   <Form autofill="debit-card"/>

2. Create a TextInput wrapper that will listen to and update values for dispatched autofill event.

   // dispatch an event with Debit Card details as a payload for "debit-card" form
   // once "John Doe Visa **72" is pressed from the dropdown, this will fill values for [month], [year] ... 
   <Form autofill="debit-card"/>
   <TextInput name="month" />
   <TextInput name="year" />
   </Form>

• Source of dropdown prototype
• Form value autofill which will be changed to an event listener

Which process will handle autofill main, renderer, or both?

https://github.com/azimgd/expensify-app/pull/1/files#diff-9276f078b750aa322d09bda48c79eb8034941df5b0f0f0c4f8b898cf9c652c41R316-R338

Autofill is handled at renderer. Data encryption happens at main.

Challenges of installing/maintaining the library you are proposing e.g. node-keytar or safestorage?

There are some challenges with node-keytar mentioned here: https://github.com/Expensify/App/issues/10107#issuecomment-1286907036

Will it be beneficial to show the native popup around for autofill? or custom popup is best to manage?

I'm not sure if we are able to implement desired functionality with a native popup.

[parasharrajat]

While I don't agree with most points being a con.

I agree. I wasn't comparing both as they are just different tools IMO. Just tried to summarize it.

[azimgd]

Demo for Autofill-able Input groups:

https://user-images.githubusercontent.com/4882133/198288410-ce15a0dd-77c1-4146-a7f8-15eb36a31c1f.mov

Usage: https://github.com/azimgd/form-autofill/blob/main/src/App.js

[ntdiary]

there will be many challenges with it. It might not work with all the forms. But yeah, we can focus on our app first and make it work.

@parasharrajat Yeah, I strongly agree, the work at hand is the most important, and I think it can have a good start and direction.😄 this is a demo variant. It's just to demonstrate autofill, so simply store the data in the indexedDB.

Autofill popup implementation.

This variant is implemented by js + dom + css.

Autofill information storage.

Actually, I want to provide several kinds of storage to use, such as localStorage/indexedDB/keychain etc.

Which process will handle autofill main, renderer, or both?

Both. This variant mainly runs in the renderer process. Encryption or keychain storage is also handled by main process.

Challenges of installing/maintaining the library you are proposing e.g. node-keytar or safestorage?

I think both of these are not too difficult for me.

Will Onyx be a good target to store the data(encryption or not)?

I'm planning to provide a default autofill admin UI, may be similar to the image below.(of course, try to be consistent with our app style). But if we want to implement this admin popup in our app. Maybe Onyx will be useful ?

Should we show a native autofill popup or a custom?

What does a native autofill popup mean? If it is datalist, I can add some more discussion about it tomorrow. 🙂

update

Some notes on 'datalist' :

Pros

1. native render

   Cons

2. Doesn't seem to support hover to preview.
3. Doesn't support password input box.
4. Need double click to open the popup.
5. It should be designed for a single input box, not convenient to fill multiple input boxes.

In addition

Actually, this is roughly my main idea, also from chromium autofill component. We can only implement a relatively simple version. And based on this idea, there are several variants to try, such as c native or datalist or js or our react-native component.

Take this card form as an example: It contains not only a card input box, but also a password input box and even an address input box. I think it would be better to find out the form type based on the information of the input box. autofill should try to make it invisible to our components. (small changes and burdens)

I have spent a week looking for a c native solution, unfortunately I didn't find a very simple solution (but haven't given up)

for this js variant:

Here are main changes we need to make in our App:

```diff diff --git a/src/components/Form.js b/src/components/Form.js index b7b5f72edb..7cc6fa6ca7 100644 --- a/src/components/Form.js +++ b/src/components/Form.js @@ -10,6 +10,7 @@ import * as FormActions from '../libs/actions/FormActions'; import * as ErrorUtils from '../libs/ErrorUtils'; import styles from '../styles/styles'; import FormAlertWithSubmitButton from './FormAlertWithSubmitButton'; +import * as ComponentUtils from '../libs/ComponentUtils'; const propTypes = { /** A unique Onyx key identifying the form */ @@ -106,6 +107,8 @@ class Form extends React.Component { return; } + // todo: trigger auto save + // Autofill.fireAutoSave() // Call submit handler this.props.onSubmit(this.state.inputValues); } @@ -218,7 +221,7 @@ class Form extends React.Component { contentContainerStyle={styles.flexGrow1} keyboardShouldPersistTaps="handled" > - + {this.childrenWrapperWithProps(this.props.children)} {this.props.isSubmitButtonVisible && (

And this is library entry file. It works by listening to events (We can also use custom events if needed). Now, It can parse the target element name to display the related form popup. For the autosave, there will be further optimizations.

But I'm not sure if you're interested in this way, can you share with me your opinion? Thanks 🙂 If this can't be accepted, I think I can propose another variant of our app component?

This is the demo video:

https://user-images.githubusercontent.com/8579651/198341092-a986d28c-f41d-4108-812f-8db2f060e4f2.mp4

[parasharrajat]

Thanks for the updates. I will check them shortly.

[azimgd]

@parasharrajat

Note: A universal solution which can be applied to normally any autofillable form will be better. For now, this issue focuses on Add debit card and login forms.

I created an NPM package with that enables form autofill on any <TextInput /> based on the Proposal of mine and Feedbacks given above.

https://github.com/Expensify/App/issues/10107#issuecomment-1277507900 https://github.com/Expensify/App/issues/10107#issuecomment-1275606422 https://github.com/Expensify/App/issues/10107#issuecomment-1292658612 https://github.com/Expensify/App/issues/10107#issuecomment-1293477763

It allows grouped form input persist and autofill. I'm planning on adding both KeyTar and SafeStorage providers.

Implementation

https://github.com/azimgd/form-autofill https://www.npmjs.com/package/electron-form-autofill

Usage at E/App

https://github.com/azimgd/expensify-app/pull/2

// just wrap your form and input with this HOC
<FormWrapper name="authentication">
  <InputWrapper>
    <TextInput ... />
  </InputWrapper>
</FormWrapper>

Demo

https://user-images.githubusercontent.com/4882133/198690581-b323a358-17d3-4852-a6a7-74bf75f21954.mov

[ntdiary]

Hi, I want to clarify a bit more, the demo code above just shows my basic process idea, and it will have a better internal structure after reorganization.

On the other hand, it has good extensibility, compatibility and api stability, I'm also planning to use Web components to avoid code collisions. These mean: (includes long-term plans)

1. It will easily support address autofill, and support fuzzy matching through RegExp, like chromium.
2. It will easily support frameworks like React, Angular, Vue, pure html/js and more.
3. If we are lucky enough to find a better native solution in the future, we can easily migrate it.
4. It can easily provide export and import functions for synchronization between browsers, 1Password, etc. (Maybe someone will like it.)
5. All of these require only minor changes or even no changes to our app!

And I can always be responsible for maintaining its code quality and performance.

[michaelhaxhiu]

Can we respond to the latest inquiries here @parasharrajat (today or tomorrow)?

[parasharrajat]

Definitely, Looking at the new comments and code samples now.

[parasharrajat]

I will share the update in sometime.

[ntdiary]

I have made some internal changes in my repo, nothing major changes in the usage. (uses web components. Encryption has not been added)

Expensify code:

```diff diff --git a/package.json b/package.json index 53c0ccf7c2..e1ed67d3d7 100644 --- a/package.json +++ b/package.json @@ -38,6 +38,7 @@ "test:e2e": "node ./e2e/testRunner.js" }, "dependencies": { + "@electron-autofill/renderer": "latest", "@expensify/react-native-web": "0.18.9", "@formatjs/intl-getcanonicallocales": "^1.5.8", "@formatjs/intl-locale": "^2.4.21", diff --git a/src/components/Form.js b/src/components/Form.js index b7b5f72edb..25fab1e0a6 100644 --- a/src/components/Form.js +++ b/src/components/Form.js @@ -10,6 +10,7 @@ import * as FormActions from '../libs/actions/FormActions'; import * as ErrorUtils from '../libs/ErrorUtils'; import styles from '../styles/styles'; import FormAlertWithSubmitButton from './FormAlertWithSubmitButton'; +import * as ComponentUtils from '../libs/ComponentUtils'; const propTypes = { /** A unique Onyx key identifying the form */ @@ -106,6 +107,8 @@ class Form extends React.Component { return; } + // todo: trigger auto save + // Autofill.save() // Call submit handler this.props.onSubmit(this.state.inputValues); } @@ -218,7 +221,7 @@ class Form extends React.Component { contentContainerStyle={styles.flexGrow1} keyboardShouldPersistTaps="handled" > - + {this.childrenWrapperWithProps(this.props.children)} {this.props.isSubmitButtonVisible && (

please let me know if you have any questions. 🙂

[parasharrajat]

Quick feedback: @ntdiary Although I agree with your motive for creating a framework-agnostic solution, it will be complicated for the team to maintain. Most folks in the team, work on React so we prefer to react.

In the end, this is a solution for our app first.

---

BTW, I will discuss this internally too.

[parasharrajat]

These are currently being reviewed.

[ntdiary]

@parasharrajat Thanks for your feedback. 😄 Maybe I pay too much attention to a universal solution, and think I can always maintain this library. I'll also consider whether I can write it with react if possible.

[parasharrajat]

Started an internal discussion. https://expensify.slack.com/archives/C02NK2DQWUX/p1667498373237429

[parasharrajat]

Waiting on the discussion before I share the next update here. It might take a day.

[parasharrajat]

I tried to save a simple text on the keychain service login and get it back which worked but I can't find it in the keychain access app anywhere. (I saved and retrieved the data in one go without closing the app). Thoughts?

[azimgd]

Using keytar ? On osx ?

[parasharrajat]

Yeah, keytar on Mac in the main process.

---

Question: Can we utilize the iCloud chain to save/get the passwords? This should allow us to share the autofill login creads across devices.

[azimgd]

Could you try sorting by "Date Modified" ? Depending on implementation it most likely saves as "New Expensify / Chromium Safe Storage"

[azimgd]

Can we utilize the iCloud chain to save/get the passwords?

It should theoretically be possible, but most likely would require us to generate our own build of chromium:

• https://chromium.googlesource.com/chromium/src/+/lkgr/crypto/apple_keychain_ios.mm#31 (here is a demonstration how attributes are set on ios)
• https://developer.apple.com/documentation/security/ksecattrsynchronizable?language=objc (api to be used for AppleSecKeychainItemRef attribute)

[parasharrajat]

require us to generate our own build of chromium:

Not necessarily. We can code our own custom node module. But it is fine if this is complicated.

[parasharrajat]

Still in discussion. Soon we will have some updates.

[ntdiary]

Update

I'm also adding a react version that will have similar changes to our app as my previous proposal. 🙂 https://github.com/ntdiary/App/pull/1/files (just a demo, still needs to be improved)

If there are still concerns, I can continue to optimize both proposals and make the code conform to our specifications.

video:

https://user-images.githubusercontent.com/8579651/200003480-904983e7-009c-47e3-a2c5-a27aa0151ec5.mp4

[parasharrajat]

I would say let's hold off doing any further analysis on this issue until we have a conclusion on the discussion. Thanks.

[michaelhaxhiu]

Please don't spend any further time providing proposals for this GH. I will provide an update as soon as I can.

[iwiznia]

Not overdue, Hax will post an update here shortly.

[michaelhaxhiu]

Will have something by tomorrow to share here.

[michaelhaxhiu]

Hey everyone. In recent weeks, we’ve discovered that there’s an opportunity to enhance and improve our password security to a higher degree by moving to a passwordless design. This new design is quickly underway and being led by internal engineers.

We do things dynamically at Expensify, meaning sometimes we pivot or change direction when better ideas come to fruition. Unfortunately this password-related job has become obsolete due to our future plans. But we still want to maintain fairness and compensate the contributors who were most actively involved. We really appreciate your pragmatism and time, and invite your continued involvement in other hard-to-solve jobs.

We did a careful review on our side and are going to pay 25% of the total job bounty to the following contributors:

• $8000 to @azimgd
• $8000 to @ntdiary
• $8000 to @parasharrajat

We feel this payout is the most fair approach since the solution was explored at length in the “proposal phase” of this job. When you have a moment please apply to the following upwork job - https://www.upwork.com/jobs/~01a9e983bc39d24141. Then I'll pay & close this job, so we can shift our #focus to other high priority jobs.

Thanks all!

[parasharrajat]

I would suggest publishing your solution as an open-source library. They might be useful to the community.

[michaelhaxhiu]

Fantastic idea to add @parasharrajat. It could be valuable to your portfolios of work in the future!

[ntdiary]

have applied, very much looking forward to this new passwordless design. 😄

[michaelhaxhiu]

@ntdiary and @azimgd are paid!

And lastly, @parasharrajat will get paid when he's ready. He asked for a few days to rotate his bank account in upwork and will message me when he's ready - 👍.

[michaelhaxhiu]

I'm going to close this GH down to ensure we are focused on the right thing. @parasharrajat see my last post above to close the loop for your payment.