Closed mne-4d closed 5 years ago
se1exin
commented
5 years ago Putting my hand up to work on this one. @perry-mitchell has there been any work on the Android side yet? Just want to make sure that any required bridge implementations expose themselves uniformly across Android and iOS.
se1exin
commented
5 years ago So I've spent the last couple of hours prototyping iOS autocomplete and have come upon a pretty significant limitation that will need some discussion before going forward.
Some background first: To enable autofill in iOS, the app must implement an AutoFill Credential Provider Extension (1). On iOS all Extensions run in a container (sandbox) separate to the main application, and are therefore treated for the most part like a separate mini-app. A part of this sandboxing is the Application's memory and stored data - just like normal Applications installed on the same device, the Extension cannot directly access the main application's data, and vice-versa. Apple does however provide a mechanism to get around this - using "App Groups" to associate the Extension and the main app, and using NSUserDefaults as the middle-man data store (2).
Now the for the limitation: React Native's AsyncStorage does not use NSUserDefaults and therefor is not usable as a interface to retrieve data (ie Buttercup Archives) from the main application. I may be incorrect, but I have spent several hours researching, including reading Facebook's source for AsyncStorage, and I think the overall consensus agrees with this conclusion. I have also tested this in the Buttercup RN projects, and have successfully loaded the Buttercup React Native JS project in my AutoFill Extension, but indeed AsyncStorage cannot and does not access the main app's data.
The only real way I can see around this is to replace the AsyncStorageInterface (in source/compat/AsyncStorageInterface) on iOS with an implementation that uses NSUserDefaults (or more secure iOS Keychain - see below) as the storage backend. This would require a fair bit of native iOS code, luckily looking around online I can see one possible open-source solution - react-native-default-preference, which would just need some wrapping to implement the buttercup-core StorageInterface. There is also an incomplete attempt that may/may not gleam some inspiration as well: react-native-shared-group-preferences.
To make things more complicated, both AsyncStorage and NSUserDefaults are not considered secure. I can see (by dumping the Documents folder for the app from the simulator) that some (the important/sensitive) data is encrypted before being sent to AsyncStorage and written to disk, however given the context of the app it may be preferred to go all-the-way and store everything using the iOS Keychain as the storage backend. This method would also support sharing data between the main app and the Extension (3), although it would involve the same process as the paragraph above. An open source RN Module that conforms to AsyncStorage exists which could make this relatively simple to achieve: react-native-secure-storage.
As you may be thinking, replacing AsyncStorage with another storage backend is not going to be as simple as it sounds.. there a few use cases that need to be considered and handled:
AsyncStorage and into the new storage backend on first launch after an app update.
AsyncStorage (via source/library/storage), but I haven't had a chance to dive into how this works or what the implications are of changing it as well, or if it is even required in the AutoFill Extension.StorageInterface to Buttercup ArchiveManager. I've only had a quick read of Google's Android AutoFill API docs, but it does not seem to affected by this problem (I may be wrong though).Sorry for the wall of text, keen to hear people's thoughts.
Relevant Links: 1) Apple AuthenticationServices implementation overview: https://developer.apple.com/documentation/authenticationservices?language=objc
2) Notes on sharing data between app containers (Scroll to the Sharing Data with Your Containing App section): https://developer.apple.com/library/archive/documentation/General/Conceptual/ExtensibilityPG/ExtensionScenarios.html#//apple_ref/doc/uid/TP40014214-CH21-SW1
3) Using Keychain instead of NSUserDefaults: https://stackoverflow.com/questions/26693913/share-between-an-ios-extension-and-its-containing-app-with-the-keychain and http://swiftandpainless.com/ios8-share-extension-with-a-shared-keychain/
perry-mitchell
commented
5 years ago Wow, thanks for the deep dive! Incredible that you've uncovered so much.
The only real way I can see around this is to replace the AsyncStorageInterface (in source/compat/AsyncStorageInterface) on iOS with an implementation that uses NSUserDefaults (or more secure iOS Keychain - see below) as the storage backend.
The storage backend would need to be handled differently on Android
Actually, I was thinking that we could simply use the new storage just for the sharing of credentials between the app and extension.. What do you think? The current data stored using AsyncStorage is just stubs for the archive connections and offline content. Sure, these too could be moved in to keychain storage but couldn't that be done separately? Ultimately it'd be good to try and avoid user data wipes as connecting archives on mobile is still more troublesome than on desktop.
If we do go the way of moving everything into keychain storage, we should also write a migration snippet to move existing vaults over to the new storage on startup.
An open source RN Module that conforms to AsyncStorage exists which could make this relatively simple to achieve: react-native-secure-storage.
This looks great! I've stumbled on it before.. Nice that it supports both platforms too.
The storage backend would need to be handled differently on Android
We could handle android as a second port of call.. Let's get this working on iOS first and then we can do another release for android.
So @se1exin what do you think the extension would look like? Just simple iOS native code? Is there any need for 2-way data flow here, or can the parent app simply write credentials into the storage for the extension to provide as auto-fill content?
se1exin
commented
5 years ago Thanks for the feedback @perry-mitchell !
Actually, I was thinking that we could simply use the new storage just for the sharing of credentials between the app and extension.. What do you think?
I had that thought as well but can see an edge case that could fail to ensure autofill credentials are in-sync with the user's other devices at the time that the autofill extension is invoked. For example, if I create/update a password on my PC, then go to login to the service on iPhone but do not open the Buttercup App first, the new/updated password will not have been synchronised to intermediate datastore and will not be available in autofill. If that makes sense? We could of course show a message or similar to educate the user that they need to do this first, but personally I think doing so is a bit clunky/not a great user experience.
That being said... I can also appreciate user benefit vs development time and introduced complexity/risk, and in this case I am very much on-the-fence about how worthwhile it would be to go all-the-way with a new storage backend to mitigate an admittedly infrequent edge case. Perhaps this could just be chalked up to technical debt so the feature can be delivered quicker; you can always go the whole-hog later on if it does turn out to be an annoyance to users. ..I leave that decision to you guys 😉 I'm happy to build it either way.
So @se1exin what do you think the extension would look like? Just simple iOS native code?
I actually wanted to discuss this in my last post, but it was well long enough already 😅. How I see it, the user will be displayed an overlay with a list of potential logins (I see it much like the browser extension login suggestions) with credentials matching the service ID (e.g. domain name) sorted to the top. Tapping a credential closes the overlay and executes the autofill. The AutoFill Extension API also has the ability to show a single credential directly in the iOS keyboard in the top bar, but I was not able to get that to work in my prototyping - I think it would be awesome to see that work though, for when there is a single direct matching credential.
This is all assuming that we use one-way data flow with an intermediate datastore, as the credentials will be available to the extension when it is invoked. Going down the two-way dataflow path, we'd need to ask the user to unlock their archive as an additional first step in the overlay, and would likely loose the ability to show a single match directly in the iOS keyboard without the need for the overlay.
As for UI/Design of the overlay, I was thinking I'd use the existing Buttercup RN Components and listing styling etc (as the overlay will still be React Native - not Native iOS), and start sending some screenshots etc once that part was underway.
Keen to hear your feedback on all of the above @perry-mitchell
tcodes0
commented
5 years ago Just getting started on the project but I was reading this issue. My understanding is that we'd like to integrate buttercup with the iOS autofill (I'm not sure if it is browser independent or safari only)
Note passwords on top of the keyboard. On my device after tapping that I have to use touchID and after I see my saved passwords in a web-page-looking screen (couldn't screenshot). I agree this would be an awesome feature to have.
After user taps a credential on the top of the keyboard, there's no view available in iOS to show the passwords and we'd have to make our own?
perry-mitchell
commented
5 years ago I had that thought as well but can see an edge case that could fail to ensure autofill credentials are in-sync with the user's other devices at the time that the autofill extension is invoked. For example, if I create/update a password on my PC, then go to login to the service on iPhone but do not open the Buttercup App first, the new/updated password will not have been synchronised to intermediate datastore and will not be available in autofill.
Yeah, this makes sense. I don't really like the idea that the credentials available from the extension could be out of sync.. So perhaps it'd be better if we could simply design it with this in mind.
Of course that means that we need to bake the functionality that allows unlocking vaults and searching for entries into the extension.
How I see it, the user will be displayed an overlay with a list of potential logins (I see it much like the browser extension login suggestions) with credentials matching the service ID (e.g. domain name) sorted to the top. Tapping a credential closes the overlay and executes the autofill.
Yep, this sounds ideal to me. Obviously with a react-native generated interface too I guess.
The AutoFill Extension API also has the ability to show a single credential directly in the iOS keyboard in the top bar, but I was not able to get that to work in my prototyping - I think it would be awesome to see that work though, for when there is a single direct matching credential.
Yep, this definitely sounds like something we'd want also. It sounds incredible!
This is all assuming that we use one-way data flow with an intermediate datastore, as the credentials will be available to the extension when it is invoked. Going down the two-way dataflow path, we'd need to ask the user to unlock their archive as an additional first step in the overlay, and would likely loose the ability to show a single match directly in the iOS keyboard without the need for the overlay.
Well here's the thing.. If we're using the keychain, could we not just write the credentials (upon each unlock) into the keychain storage so they're immediately available? We could provide a way to unlock and update as well, perhaps.. And for ease of use we could force users to use touch/face ID to unlock (not accepting a password-based unlock process).
There's a lot of choices here on the design of the extension but I'm sure many will end up not being possible. I've got 0 idea of how this will look once it's complete but I just have a bad feeling there'll be some number of brick walls 😅
As for UI/Design of the overlay, I was thinking I'd use the existing Buttercup RN Components and listing styling etc (as the overlay will still be React Native - not Native iOS), and start sending some screenshots etc once that part was underway.
Well what would be ideal would be the use of our current RN components in the new extension. The encryption code is handled by an iOS/Android-JNI bridge and some native code that calls a Rust binary.. It's not super portable. It'd be great if the extension could literally be the same app, just with another entry point or something to that effect. No idea if this is possible or whether this might breach some memory constraints.
@se1exin Great stuff! This is exciting.
perry-mitchell
commented
5 years ago Note passwords on top of the keyboard. On my device after tapping that I have to use touchID and after I see my saved passwords in a web-page-looking screen (couldn't screenshot).
@Thomazella I wonder if this is due to the touch-unlock requirement for keychain access? I'd suppose so. I guess those passwords would simply be the ones already stored by the OS (unless you're using another password manager).
After user taps a credential on the top of the keyboard, there's no view available in iOS to show the passwords and we'd have to make our own?
Yep, we'd need to build the UI to provide a sort of portal into the Buttercup app (via the extension we're discussing).
tcodes0
commented
5 years ago I guess those passwords would simply be the ones already stored by the OS (unless you're using another password manager).
@perry-mitchell yeah. I was just using stock safari. I had some saved passwords for some sites.
It'd be great if the extension could literally be the same app, just with another entry point or something to that effect.
just an ideia: Register buttercup:// url to open app on ios. Somehow differentiate from a regular app open (native code in appDelegate.m maybe). then:
app.js
render(){
// if indirect open, return extension
// return buttercup
}I've just spent a few more hours playing around with one and two-way data flow solutions (for lack of better terms), and have some more findings...
Note: These are just my findings, it would be awesome if there is anyone out there that has worked with the iOS AutoFill Extension before, and that could lend some additional insight!
One-way data flow
If we're using the keychain, could we not just write the credentials (upon each unlock) into the keychain storage so they're immediately available?
I've tested adding an additional step in shared/archiveContents/updateCurrentArchive(), that syncs the Archive into the iOS Keychain when an archive is first unlocked and also when any change is made. I am then able to successfully extract the archive from the Keychain from within the AutoFill Extension - so one-way data flow over the Keychain works! Futhermore, Keychain entries can be optionally protected with Password or Biometrics. In my testing I have left Password/Biometrics off, and just ensuring the device has been unlocked (which it needs to be anyway for the user to be accessing a password protected service) and it works well. The Keychain entries are sandboxed to just the main app and the autofill extension, so no other apps etc can possibly access it.
Two-way data flow
Of course that means that we need to bake the functionality that allows unlocking vaults and searching for entries into the extension.
I've also tested replacing AsyncStorageInterface with a new "SecureStorageInterface", which wraps the react-native-secure-storage library - and it works perfectly! I've tested adding an Archive in the main app, and then loading the entire RN JS app in the AutoFill extension, and the Archive references are successfully listed in the app start page. Tapping an Archive starts to decrypt it, but...
...the Crypto bridge is not available in the AutoFill Extension, as iOS does not allow direct access to the main app's native code. We can however work around this by exporting the Crypto bridge and related native code as a Framework, and then import the Framework into both the main app and AutoFill Extension Xcode Targets (the only allowed way to share code between Apps/Extensions). I'm also not sure at this point if we are able to pull down an Archive over the network in the Extension. Additionally, a few UI elements are blocked in the Extension by iOS, such as ActionSheets and Alerts, as the Extension is not actually a fully-fledged iOS application, however these can easily be just omitted from the Extension UI.
Of course that means that we need to bake the functionality that allows unlocking vaults and searching for entries into the extension.
If we're using the keychain, could we not just write the credentials (upon each unlock) into the keychain storage so they're immediately available? We could provide a way to unlock and update as well, perhaps.. And for ease of use we could force users to use touch/face ID to unlock (not accepting a password-based unlock process).
Unfortunately, these are all pretty much in conflict with each other if AsyncStorage is kept as primary Archive Source storage, as nearly everything that ButtercupCore puts into AsyncStorage will need to be replicated into the Keystore for the Extension to have access to unlock. And if we're doing that then it doesn't make much sense to also write the unlocked credentials into the Keychain as well. I really like how simple it was to switch out AsyncStorage with react-native-secure-storage rather than add additional code to replicate to the Keystore, but as mentioned unlocking Archives in the Extension has the side effect of requiring the Crypto bridge to be pulled out of the main app and converted into a Framework, and I'm still not even sure if network based Archives will even work (they might - but given all the other restrictions I'm not assuming anything).
..So where to from here? To me there are one of two distinct paths that can taken:
1) Go the simple route of one way data-flow by copying unlocked Archives into the Keychain when the main app is in use. The user will just need to be informed that their credentials are read-only and the main app will need to be opened to reflect any recent changes to their Archives. This will be by far the faster solution, but at the cost of a limited user experience.
2) Migrate AsyncStorage to react-native-secure-storage for iOS (along with migrating existing users), convert the Crypto Native code to a Framework Project, and test that remote Archives will work. Obviously a much larger body of work, but should put in place in all the necessary pieces to ensure the best possible user experience, and also allow creating/updating of passwords directly from Safari/other apps later down the track.
Obviously with a react-native generated interface too I guess.
I've been checking out the Search functionality merged in #130 and the work done there should serve as the perfect base for the main Extension UI.
It'd be great if the extension could literally be the same app, just with another entry point or something to that effect
Spot on - the extension JS would be another entry point exposed via React Native's AppRegistry, that way we can strip as much as we don't need from the main app. E.g. a new index.autofill.ios.js file as the Entry Point.
@Thomazella
just an ideia: Register buttercup:// url to open app on ios. Somehow differentiate from a regular app open (native code in appDelegate.m maybe).
Unfortunately iOS's AutoFill Extension API is a bit more complicated than that.. but if you've read this far you probably don't need me to tell you that now 😂
@perry-mitchell
I just have a bad feeling there'll be some number of brick walls
Yep, this is starting to feel a little bit like a maze 😛..
se1exin
commented
5 years ago
...the Crypto bridge is not available in the AutoFill Extension, as iOS does not allow direct access to the main app's native code. We can however work around this by exporting the Crypto bridge and related native code as a Framework, and then import the Framework into both the main app and AutoFill Extension Xcode Targets (the only allowed way to share code between Apps/Extensions). I'm also not sure at this point if we are able to pull down an Archive over the network in the Extension.
So it looks like I just needed to sleep on it. I jumped in this morning and was able to easily link the Crypto Bridge into the Extension without pulling the Crypto Bridge code out of the main project, and whola the Extension is able to decrypt Archives, including pulling them over the network. Two-way data flow works 🎉
I've started moving ahead with two-way data flow and have already implemented Keychain based storage, including auto migration of data from AsyncStorage into Keychain storage for existing users. Also, the react-native-keychain module (used in the TouchID functionality) conflicts react-native-secure-storage at compile-time, so I have updated touchUnlock.js to work with react-native-secure-storage. The changes are backwards compatible and no migration for existing users is required.
My work so far is in the branch issue/125 in my fork: https://github.com/se1exin/buttercup-mobile/tree/issue/125.
TODO List:
touchUnlock.js by removing react-native-keychain, react-native-storage, and related supporting codereact-native-keychain to react-native-secure-storageA quick preview to demonstrate Keychain storage, Cyrpto, and network Archives working and being shared successfully between the main app and the extension:
perry-mitchell
commented
5 years ago I jumped in this morning and was able to easily link the Crypto Bridge into the Extension without pulling the Crypto Bridge code out of the main project, and whola the Extension is able to decrypt Archives, including pulling them over the network. Two-way data flow works 🎉
This made my week.. Awesome news!
A quick preview to demonstrate Keychain storage, Cyrpto, and network Archives working and being shared successfully between the main app and the extension:
This is nuts.. well done! You've made incredible progress already. Definitely can't wait to give this a test run.
I completely understand that it may still be early stages, but what is your plan with the "Passwords" button above the keyboard? Should it open some smaller frame to display something from the app? Just wondering about usability when opening the whole Buttercup app for a password. I haven't honestly used this feature with another password manager so I'm a total novice when it comes to how Apple have implemented it.
Loving the progress here! 🚀
tcodes0
commented
5 years ago @se1exin wow, awesome! Didn't fully understand the gif: after tapping passwords on the top of the keyboard, it invokes the extension? If so, whatever it used to show before user installs buttercup is still accessible? This is definitely a good step forward :D
sallar
commented
5 years ago This is incredible! Great work guys I can’t wait to try this in action 🚀🥳
se1exin
commented
5 years ago Thanks for the positive feedback everyone 😎
Didn't fully understand the gif: after tapping passwords on the top of the keyboard, it invokes the extension? If so, whatever it used to show before user installs buttercup is still accessible?
Yeh sorry the gif isn't that helpful, I could only record 30s with my gif recorder program so had to rush through the process.
Tapping a username/password field brings up the iOS keyboard as per usual, if the user has Autofill enabled (in iOS settings) then iOS will add the 'Passwords' button above the keyboard - this is all handled by iOS outside of Buttercup.
Tapping the 'Passwords' link asks iOS for installed Apps with AutoFill functionality, and if Buttercup is enabled as an AutoFill provider (in iOS Settings), iOS invokes the Buttercup AutoFill Extension. Again, this is all handled by iOS. At this point I can only assume iOS will also display any other items you may have in your Keyboard QuickBar, but I don't actually have other things in my QuickBar, so I'm unable to test. If your able to suggest a test scenario I'd be happy to try and reproduce and test what happens (e.g. "Install App X and enable feature Y to add new item to iOS Quickbar").
Should it open some smaller frame to display something from the app? Just wondering about usability when opening the whole Buttercup app for a password.
Good question. In the demo gif I am loading the entire existing JS App into the extension frame purely to demonstrate that it is possible to load and display an RN JS App in the autofill extension environment. The only reason it is showing the entire Buttercup app is because I have just simply not gotten to the step of building a dedicated UI for it. I think I mentioned it in a previous comment, but the plan at the moment is to create a new RN entry point based on and similar to the Search Screen from #130, but...
...I'm a very keen to hear feedback on what people think the Extension frame UI should have and should look like. Hopefully over my last few comments I have successfully outlined the many constraints that have to be considered, but seeing as they are quite dense here is a summary:
--
Edit: Nevermind - just figured out how to show a single direct match as well!! More details and hopefully a demo soon!
perry-mitchell
commented
5 years ago the plan at the moment is to create a new RN entry point based on and similar to the Search Screen
Yeah this sounds excellent. Definitely a great idea to perform a quick domain search using the one provided by iOS. Ideally we'd have some combination of the following:
🤯
se1exin
commented
5 years ago A quick update, direct AutoFill from the iOS Keyboard QuickBar is now working 🎉.
FYI the Touch/FaceID prompt that is shown after the credentials is activated by iOS, not Buttercup, so I don't think we can exclude it.
Ideally we'd have some combination of the following:
- Results for the domain
- Search results
- Tree-based browsing as in the app (?)
Thanks @perry-mitchell. The UI is now all that is left to do, so will get started with it and work on your 3 points in that order.
sallar
commented
5 years ago This is amazing man! Great work wow
perry-mitchell
commented
5 years ago 
perry-mitchell
commented
5 years ago Hi @se1exin! Hope all is going well mate. Please let us know if you need a hand with anything. If there's something that we could add in the core to be of more use we can also do that quite quickly.
se1exin
commented
5 years ago @perry-mitchell thanks! I've been chipping away at it over the week, but it's also been the first week back at work, so it's been a busy one! I should have a PR ready over this weekend though.
Will shout out if I need a hand with anything!
Edit: The UI is nearly complete with results for domain and search results working. Last thing to do is handle error cases such as failed Touch ID or no archives with Touch ID at all. For no Touch ID at all I was thinking of failing back to manual unlock of Archives via an adapted ArchivesList page.
Demo of UI so far. Search works as well but had to keep the gif short 😉:
perry-mitchell
commented
5 years ago Wow! It just keeps getting better. I knew this feature would be great but I didn't have any idea it would be this powerful. Definitely a feather in the cap of mobile password managers.
Thanks for seeing this through. Definitely agree regarding work - I also just got back, and it's manic 😅
se1exin
commented
5 years ago @perry-mitchell the PR is ready 🤘
perry-mitchell
commented
5 years ago Released in 1.8.0.
perry-mitchell
commented
5 years ago We also published a post on the feature here (credit given @se1exin :wink:)
Hi devs
Would it be possible to make use of ios password autofill implementation into buttercup ? Some other password manager did and this functionnality is a game changer in ios ecosystem.
congrat for everything, very nice project