OAuth login to client not working in Android WebView (and iOS too?)

[hwasiti]

When I use Android Webview to browse inside an android app and load the hypothesis bookmarklet, it will succeed in loading the side bar and showing the public annotations. However, I have to login to do my annotations.

When I click on login, and because Webview cannot open a new tab or window, it will open the login page in the same "tab" or Webview "window". After I enter my credentials, it will redirect me to a blank page and will stuck there.

I tried to investigate what is happening when it redirects to the blank page. Opened: http://google.com then loaded the bookmarklet: javascript:(function(){window.hypothesisConfig=function(){return{showHighlights:true,appType:'bookmarklet'};};var d=document,s=d.createElement('script');s.setAttribute('src','https://hypothes.is/embed.js');d.body.appendChild(s)})();

and login with my hypothesis credentials.

The title of the blank webpage after login is: Authorization completed

And the url: https://hypothes.is/oauth/authorize?client_id=631206c8-7792-11e7-90b3-872e79925778&origin=https%3A%2F%2Fhypothes.is&response_mode=web_message&response_type=code&state=d293c1b2c1829b4f

It seems it fails to go back to the original url (here: http://google.com).

In Android Chrome browser and other browsers which are all successfully working with the bookmarklet, what happens is that when I click on login, a new tab will popup and when I finish login this window will close and return to the former window with the bookmarklet active.

--> ### Steps to reproduce

1. Android Webview in an Android app (Android 6.0.1, SM-N9005)
2. Load a web page
3. Load the hypothesis bookmarklet
4. Login using hypothesis credential
5. Redirects to a blank page

### Expected behaviour It should redirect to the original website that I want to annotate with bookmarklet active and logged in

Actual behaviour

Redirects to a blank page titled Authorization completed and stuck there URL: https://hypothes.is/oauth/authorize?client_id=631206c8-7792-11e7-90b3-872e79925778&origin=https%3A%2F%2Fhypothes.is&response_mode=web_message&response_type=code&state=d293c1b2c1829b4f

### Browser/system information Android Webview (Android 6.0.1, SM-N9005) User agent: Mozilla/5.0 (Linux; Android 6.0.1; SM-N9005 Build/M4B30X; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/61.0.3163.98 Mobile Safari/537.36

Changing the user agent of the Webview did not help: For example I tried user agent: Dalvik/2.1.0 (Linux; U; Android 6.0.1; SM-N9005 Build/M4B30X)

and tried: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36

[hwasiti]

Does the new OAuth authentication have been applied already?

Just found this in hypothesis blog published 1 week ago: Leaving Cookies for OAuth Authentication https://web.hypothes.is/blog/leaving-cookies-for-oauth-authentication/

I think my issue related to this quote from the blog post:

From a user’s point of view, little has changed. We have moved login using username and password out of the sidebar and into a popup window, which can be used not only by official Hypothesis clients but also by registered third-party clients.

[hwasiti]

To make it easy to test this, I made a simple Android app that you can use to test this issue.

This is the .apk file that can be installed on an Android device or using a free version of Genymotion Android Emulator :

https://www.dropbox.com/s/gqgemclw3yn1sax/dummy_copy.apk?dl=1

I made few tests in several websites using this app. Take for example http://www.stackoverflow.com or http://www.quora.com I used login using Facebook and in Quora login using Google and had no problems with them in the Android WebView. So it seems that there is a way to handle OAuth in Webview on other websites.

It will be great to address this issue in hypothes.is bookmarklet. This will open the door to all mobile apps where the only way to browse is through Webview component. I checked that this is affecting IOS apps as well.

[robertknight]

Thank-you for the report @hwasiti and the detailed research.

Can you tell me what the original scenario was where you encountered Hypothesis inside a web view? It's nice to know the context/use case associated with a bug report if it isn't confidential.

[robertknight]

According to this Stack Overflow answer in cases where the code of the Android app that is displaying the web view can be altered, it is possible to support popup windows with some additions to the Android app itself.

However, this won't help in cases where the web page that embeds Hypothesis is being displayed inside an app maintained by a third party.

[robertknight]

As I understand it, Android and iOS provide two different classes of embedded browser for apps to use:

1. An embedded web view which does not share state with the main OS browser - WKWebView and Android WebView
2. An "embedded" version of Safari (SFSafariViewController) or Chrome (Chrome Custom Tabs) that shares state (eg. cookies) with the host browser, but provides less control to the embedding app.

It would be useful to understand whether this issue applies to both of these types of browser, or only one of them, or if the answer is different depending on whether the platform is iOS or Android.

[hwasiti]

@robertknight Hi Robert,

Thanks for looking into this!

The answer in Stackoverflow shows how to handle a popup in Webview, however still it is not clear how to handle passing the JSON OAuth data to the main page after closing the popup.

The blank page source is the following (It is the page that should be closed and will return OAuth user's data to the main page in order to make a login in the bookmarklet):

Now the JS script 'post-auth.bundle.js' will pass the JSON data above that has been received to the main window to login. The main code in this script that is of interest here is this:

The problem is that with the Stackoverflow workaround the new Webview will not see the original Webview as the 'window.opener'. So this Javascript will fail to pass the JSON data to the original Webview.

I tried to modify the Javascript 'post-auth.bundle.js' and make my own, but it needs other librararies in hypothes.is server, and it is too hacky, and will break with any change to the Authentication system of hypothes.is.

If I find a way to:

1. take the JSON data from the 'Authorization complete' blank page
2. loading the main page again in the same webview with the bookmarklet
3. let my script click on hypothesis bookmarklet login link, but blocking the popup
4. pass the JSON data manually by my own Javascript into the login waiting bookmarklet to complete the login

However, I was not yet successful to do the last step. Currently trying in Desktop browser to do it, and if succeed I will implement it on Android.

Unfortunately, I do not have IOS developing tools to test it on IOS, but from what I found in Stackoverflow, it seems it has the same issue with handling popups.

Regarding the other embedded browser for apps to use (e.g., Chrome Custom Tabs), the problem is that the app has no control over this embedded Chrome Custom tabs, so it is difficult to load the javascript that will load the bookmarklet (see here). Beside the other things that the app needs to work with the Webview, in term of control or data exchange between the app and the embedded Webview. And hence the questions in Stackoverflow to use Webview even with the difficulties to handle popups.

It would be great to me and others who will face this issue in the new OAuth hypothes.is login system, if you can do it without using popups like in 'Login using Google or Facebook' in http://www.stackoverflow.com or http://www.quora.com

I used login using Facebook in stackoverflow, and login using Google in Quora, and had no problems with them in the Android WebView because they do not use popups in their OAuth implementation.

It seems there is a reason they did authentication without using popups. Browsers with popup blockers will face issues, and all apps that uses Webview will need hacky ways to handle them.

[hwasiti]

When the user makes a login in the hypothes.is website in a desktop browser: https://hypothes.is/login

then navigate to other website, and loads the bookmarklet. Why the bookmarklet still needs another login?

The diigo annotator bookmarklet for example , when I login in their website: https://www.diigo.com/sign-in?f=diigolet

then if I load any website and load the diigo bookmarklet:

javascript:(function(){s=document.createElement("script");s.type="text/javascript";s.src="https://www.diigo.com/javascripts/webtoolbar/diigolet_b_h_b.js";document.body.appendChild(s);})();

The bookmarklet will not need any more login.

If this was true for hypothes.is as well, then my problem will be solved. The user should login in the main website of hypothes.is (which is good that it does not need a popup) then the bookmarklet will be logged in already using cookies.

[robertknight]

Hello @hwasiti

Can you tell me a bit more about your use case? Are you building your own app with a web view that you want to use Hypothesis in, or are you using a web view in someone else's app? Are the pages you want to use Hypothesis in pages that you control or are they arbitrary web pages?

I'm sure we can find a way to enable authentication for your use case without requiring elaborate hacks. We used to use cookies exclusively for authentication in the client and for several reasons moved away from that to OAuth. Perhaps we might need to end up using a combination of the two to best handle different use cases. First however, I'd like to understand the context of the request a bit better.

[hwasiti]

Hi @robertknight

Thanks for the followup!

Can you tell me a bit more about your use case? Are you building your own app with a web view that you want to use Hypothesis in, or are you using a web view in someone else's app? Are the pages you want to use Hypothesis in pages that you control or are they arbitrary web pages?

I am developing my own Android app that I intend to publish in Google Playstore as a free Android app which uses Webview for arbitrary web pages. The user base that I am targeting hopefully will be at least 50K-100K within 12 months.

I have a sample apk that can be tested on a a free Genymotion Android Emulator . The app will load google.com and from there you can go to any website and you can use a button to load the hypothes.is bookmarklet. And you can try to make a login to test the authentication which will end to that 'Authorization completed' blank page.

I'm sure we can find a way to enable authentication for your use case without requiring elaborate hacks. We used to use cookies exclusively for authentication in the client and for several reasons moved away from that to OAuth. Perhaps we might need to end up using a combination of the two to best handle different use cases.

Yes!! That would be really great.

While a lot of OAuth authentications that I've seen used popups, but a few examples that I have mentioned (e.g., stackoverflow, Quora) managed to avoid popups and still used social login with OAuth. I don't know whether that would be easy to be done. But that's the ideal solution for the far future.

I think OAuth authentication is pretty new for hypothes.is (from the blog post I understand that it has been implemented ~2 weeks ago). Maybe implementing OAuth without popup - if it is easy to be done - will be the best way before extending the OAuth authentication into Social login and integration with organizational identity services which are the main goals of implementing OAuth in hypothes.is.

But changing the OAuth authentication certainly takes time and energy, so it will be nice, in the meantime, to support using cookies for authentication in the client beside the OAuth, until the OAuth authentication gets perfected to support all users' usage scenarios.

I believe, since implementing the new OAuth is pretty new in hypothes.is, there will be others who will raise issues like mine in the future. I appreciate your genuine interest to help..

[robertknight]

I've just done some testing on iOS using the WebView app with the following results:

• OAuth login works using SFSafariViewController
• OAuth login fails using UIWebView and WKWebView

This means that it is possible to login and comment or reply to annotations in apps like Twitter that use Safari View Controller.

Incidentally I tried to login to Quora via UIWebView and that failed in my test. I haven't yet had a chance to test on Android.

While a lot of OAuth authentications that I've seen used popups, but a few examples that I have mentioned (e.g., stackoverflow, Quora) managed to avoid popups and still used social login with OAuth.

The original "standard" way to use OAuth was via redirects rather than a popup, with the auth code being returned via a fragment parameter or query string, and I imagine that probably does work in web views. In the case of the client however that is trickier because the client is embedded in an iframe rather than being the top-level page.

Maybe implementing OAuth without popup - if it is easy to be done - will be the best way before extending the OAuth authentication into Social login and integration with organizational identity services which are the main goals of implementing OAuth in hypothes.is.

If we can find a simple solution that would be great. Realistically supporting social login is probably going to be a higher priority goal for Hypothesis as an organization than supporting the client in web views.

Since you are building the app yourself, my current thinking is that the path of least resistance is likely to be:

1. Implement support for window.open in your app (eg. via the onCreateWindow implementation)
2. Find a solution to the issue of window.opener not being available in the newly opened window.

The other option would be to figure out a hybrid of cookie & OAuth authentication. For example, the client could make a cookie-authenticated GET /api/token request when it loads if it is a "trusted" client and the service would issue a temporary access token in response. In your case presumably you can make sure that third-party cookies are always enabled. This would require changes to the client and service and a bunch of coordination amongst H devs though.

[hwasiti]

OAuth login works using SFSafariViewController OAuth login fails using UIWebView and WKWebView

With my ipad2 (ios9) I installed the same app that you have tested, I tried to test hypothesis login in the bookmarklet and all methods failed. Both (UIWebView and WKWebView) method and (SFSafariViewController) method eventually showed blank screen since it could not open a popup. What I have done is the following (the following points works without problem in a desktop browser, because it could open popups):

1. Write the URL in the address bar: hypothes.is
2. Choose SafariVC and click Go
3. Click on Get started on the hypothes.is website
4. Click on "other browsers" in the 2nd point
5. Click on Launch hypothesis This will launch the bookmarklet
6. click on Log in

And it will go to the infamous blank page. Which I think it is the very same 'Authorization complete' page.

I was curious, the only thing that we have not tested yet is the chrome custom tabs in Android. I have Android 6 (Marshmallow) with the latest Chrome browser version and latest version of Android Webview. So I subscribed to hypothesis blog in Feedly (RSS app). And opened the latest hypothesis blog post in the chrome custom tabs of Feedly. Then clicked on "h" logo which is actually: hypothes.is And followed the points above until I made login in the bookmarklet. And after filling my hypothesis credentials and clicked login, and again: BLANK page. Twitter, Facebook and many other apps which uses Chrome custom tabs in Android or SFSafariViewController in IOS are ALL affected.

That means hypothesis has problem in Login in ALL apps. Whatever the platform is (ios or Android) and whatever the method that the app uses.

Incidentally I tried to login to Quora via UIWebView and that failed in my test. I haven't yet had a chance to test on Android.

With my ipad2 I tried to test Quora login. I was successful in login using Google credentials in both (UIWebView and WKWebView) method and (SFSafariViewController) method. Yes it will throw an error, but ignore it and continue to the login process. You will find that you will be able to login without using any popup. And also it was successful with SFSafariViewController method even without error.

Also I tried another similar app on my ipad, and this one also could login in Quora using Google and this time without any errors using all methods.

I have re-tested Quroa login with an Android app published in Playstore (just like what you've find in IOS), and Quora login worked with Andoird Webview too.

It seems Social Login without popup is doable somehow on both IOS UIWebView and Android Webview.

Find a solution to the issue of window.opener not being available in the newly opened window.

That's what I am trying to do for the last 2 weeks. I will dedicate the whole next week for a solution, if I will not be able to do it, I'll have to give up, and perhaps search for another highlighting service or move on without highlighting feature in my app.

But even if I solve my issue (which I begin to doubt it really, after exhausting most tricks in the past 2 weeks), that will not change the fact that ALL 3rd party apps like Facebook, Twitter or whatever will face this issue with hypothesis, unless you make the login OAuth just like Social Login in Quora, Stackoverflow and many others who managed to do it without any popup. And it seems this is why they stayed away from popup during their OAuth authentications.

Edit (multiple times): for adding other test results whenever a new idea pops up in my mind. It seems I can handle popups :-)

[klemay]

Another user encountering this on Chrome and Safari for iOS: https://hypothesis.zendesk.com/agent/tickets/2273

[hwasiti]

I could make it work fine on Android webview. Here is an example : https://github.com/hwasiti/Android_Popup_Webview_handler_example

[ndungu4st]

Hello hwasiti kindly assist as i am having the same problem with my site webview app. That is, i cant login and if i login i cant logout. In addition i am using pop up for the login, after inserting my credentials it shows that its verifying but stays on the same pop up page such that i have to close the app, then after reopening it, it shows that i am login but then if i try logging out the does not log out. kindly give suggestions or assist with the codes plus which file should i add the codes that is, is it "android manifest.xml" "mainfragment.java" etc {codes on this https://github.com/hwasiti/Android_Popup_Webview_handler_example} Best regards

[hwasiti]

@ndungu4st I suggest to import my github project above into Android Studio and play with it. You can make the webview to load google.com and then you can search for hypothes.is Click on: hypothes.is Get started other browsers Launch hypothesis

You will see that the bookmarklet is loaded

then you can test whether my app will work in hypothes.is bookmarklet login and logout, and if it works how it is being done. As far as I remember, you don't need to change anything in the Androidmanifest.

The idea is to create a new dialog that contains the new popup webview when the method onCreateWindow is triggered by the javascript that wants to open a popup.

A step by step instruction that I followed in creating this app is here: https://stackoverflow.com/a/19068076/1970830

Good Luck!

[ndungu4st]

@hwasiti Thanks, will try it out

[egiesem]

fix: https://www.youtube.com/watch?v=vFSgIdtEl2k