search

openhab / openhab-android

openHAB client for Android
https://play.google.com/store/apps/details?id=org.openhab.habdroid
Eclipse Public License 2.0
598 stars 316 forks source link

Problem accessing Webview with untrusted SSL certificate #948

Open tobiasisenberg opened 6 years ago

tobiasisenberg commented 6 years ago

Actual behaviour

When trying to view Webview items with the beta app, I no longer can see Webview elements on my own server using a special port of the format: Webview url="https://my.own.domain:64443/grafana/d-solo/000000001/openhab-data?orgId=1&panelId=16&from=now-24h&to=now" height=15 I am not sure if the reason is the own domain or the special port. The same sitemap loads fine in the basicui website (both accessed locally and remotely) and the stable Android app (both accessed locally and remotely). In the beta app I sometimes get the content of another Webview item previously looked at from the same sitemap, as if that is still in the cache. Other Webview elements (not on my own domain and without a special port) also work fine, even in the beta app.

Expected behaviour

The web content should be loaded in the app.

Steps to reproduce

  1. Access the part of the sitemap that has the Webview
  2. Wait (nothing happens)

Can you reproduce the issue in demo mode?

n/a

Environment data

Client

Server

openHAB Cloud

Logs

App log

07-11 20:43:52.831 19082 19082 D AnchorWebViewClient: Did not find anchor from url https://my.own.domain:64443/grafana/d-solo/000000001/openhab-data?orgId=1&panelId=16&from=now-24h&to=now

openHAB Server log

Insert your log here

Reverse Proxy log

Insert your reverse proxy error log here
Insert your reverse proxy access log here
tobiasisenberg commented 6 years ago

This also happens to non-grafana URLs, such as those pointing to html files in config/static/*.html

tobiasisenberg commented 6 years ago

Another note: i tried on various devices (phones and tablets) and various Android versions (up to Oreo 8.1).

tobiasisenberg commented 6 years ago

Another update: this behavior still happens if the URL in the Webview item does not require a special port:

Webview url="https://my.own.domain/grafana/d-solo/000000001/openhab-data?orgId=1&panelId=16&from=now-24h&to=now" height=15 Webview url="https://my.own.domain/static/test.html" height=10 Webview url="https://my.own.domain/static/image.png" height=10

, regardless whether the URL with which the app accesses the OpenHAB instance (as specified in the settings) still uses the port (64443 in this case) or not; the app still does not show the Webview contents.

So maybe the issue is the fact that both the URL for accessing OpenHAB (as specified in the settings) and the URL in the Webview item use the same domain "my.own.domain"? Tested now with the most recent version 2.2.18-beta. The non-beta version from the Play Store shows the Webview items just fine, with or without the special port in the URLs, and with or without the special port in the URL to access the OpenHAB instance (as specified in the settings). Updated the title of the issue to reflect this information.

mueller-ma commented 6 years ago

If the webview content is on the same server than openHAB, you can use remove "https://my.own.domain" from the url.

tobiasisenberg commented 6 years ago

Interestingly, while this suggested approach of removing "https://my.own.domain" works using BasicUI in a web browser, it works neither in the stable nor in the beta Android app---the space for the webview is being created by both apps but the linked page never loads/is never embedded. Moreover, I don't see why the full URL should fail, after all it works in the stable app and I would expect all correct full URLs to work.

My hunch is that the access with the full URL fails in the beta app because the access to my "https://my.own.domain/" is, of course, password-protected and the beta app has changed something for how the password is stored/reused for accessing one's own server. Yet even a full URL with basic authentication does not seem to work correctly, then it shows only in the stable app but not in the beta app and also not in the web browser.

I thus consider this behavior a bug in the current beta app. There seems to have been a discussion about a related issue and a PR that addressed it, but I am not sure if it is exactly the same issue as in the current beta app since these discussions are from 2015.

mueller-ma commented 6 years ago

I tried to reproduce your issue:

Webview url="http://openhab.local/" height=10
Webview url="http://openhab.local/basicui/app?sitemap=demo2" height=10

Port 80 is protected with basic authentiacation and I can see both webviews. Sometimes they are in the wrong order, but only for a short time. This seems to be a problem with recycled, but uncleared, view (as we had with icons).

Is my.own.domain the same server where openHAB runs? Please note, that browsers may cache credentials and send them to the server without notifying you.

tobiasisenberg commented 6 years ago

Well, it seems that you are only accessing the OpenHAB instance from within your local network as "openhab.local" would not resolve to your external IP address from outside. I specifically want to link to "https://my.own.domain/..." because I want to be able to access my data both from within my home network and when I am not at home. I can only say that I can access any of these full URLs independently with a web browser (and authentication) from both inside and outside my home network, but when I place the same (full) URLs into a Webview tag (both with port 80 and with a special port) then the access only works for Basic UI in a web browser and in the stable app, but no longer with the beta app. I think that if the approach of using Webviews without "https://my.own.domain" for accessing local content would work not only in the web browser but in the apps as well then it would be fine (should be mentioned in the documentation, though).

I would be more than willing to help testing and debugging this issue with you using my installation to get the bug resolved, if you have specific things for me to try out (starting from coming Wednesday), in that case please contact me by e-mail.

tobiasisenberg commented 6 years ago

And, btw, I do not think that simply clearing Webviews before they are re-used will address my issue. I tried several times before to force-stop the Android apps and then clearing the cache before restarting them (which should similarly delete previous Webview contents), yet the issue that my local Webview contents was not loaded did not go away. Yet, if your change becomes part of the beta Android app I can test again.

mueller-ma commented 6 years ago

Are you using a valid certificate for your domain?

And, btw, I do not think that simply clearing Webviews before they are re-used will address my issue.

From your posts I can see that there are two issues:

  1. Webviews don't load at all
  2. Webviews are sometime in the wrong order. I sometimes get the content of another Webview item previously looked at from the same sitemap, as if that is still in the cache

My PR should fix issue 2. I can attach a debug apk for you tomorrow.

tobiasisenberg commented 6 years ago

I use a self-signed certificate which I typically accept the first time I use the Android app to access my own OpenHAB server. But you are absolutely right that your fix should address issue #2, but my issue #1 is (a lot) more important for me and I am a bit afraid that if you close this issue post then issue #1 will not be addressed. ;-) But maybe I misread the automatically generated status post of Github here which stated that your fix will close this issue ...

If you send me the APK I can test it on Wednesday.

tobiasisenberg commented 6 years ago

And to reply to another question I had previously overlooked: yes, my.own.domain runs both OpenHAB and the web sites, both using the same ports and the same authentication. In the apps I enter https://my.own.domain:64443/ as the remote server URL and the login and PW that I would also use to access the web sites and the web-based basic UI.

mueller-ma commented 6 years ago

Do you see Failed to validate the certificate chain, error: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. in the app log?

tobiasisenberg commented 6 years ago

Sorry for the delay, I was on a plane and then could not think straight due to jet lag. Now I slept and did the test with the beta app. And yes, I do see this error message:

08-22 18:37:23.265 17570 21360 I X509Util: Failed to validate the certificate chain, error: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.

Does this mean that with my self-signed certificate I am out of luck?

mueller-ma commented 6 years ago

Does this mean that with my self-signed certificate I am out of luck?

Yes, at least for now. I have to see how we can change the default certificate validation from the webview. For now you can use relativ paths in your webview.

tobiasisenberg commented 6 years ago

In that case I would like to point out again that there is a third bug here: relative path Webview widgets are shown in neither stable nor beta app (tested with most recent 2.2.19-beta), in neither local nor remote access. I only see the Webview contents when I pull up the Basic UI in a regular Webbrowser (there everything works as expected). Tested code:

Webview url="/static/image.png" height=10 Webview url="/grafana/d-solo/000000001/openhab-data?orgId=1&panelId=16&from=now-24h&to=now" height=15

Again, I'd be happy to help testing/debugging this issue so that it can be sorted out so that we can get it to work for both the browser and the app. Let me know if I need to look for, for example, specific debug messages.

mueller-ma commented 6 years ago

I'm sorry, my suggestion was wrong. For the webview itself there is no difference between absolute and relative urls. The relative urls are transformed to absolute urls before giving them to the webview. Webviews over http and https with official certificates with relative urls are fixed in https://github.com/openhab/openhab-android/commit/b596894d2c653b51638a49941ac6c08dba3c3437 (not yet in the stable release).

kleintody commented 5 years ago

I have the same problem as @tobiasisenberg: WebView is not displayed in OpenHAB Android app if it points to a https-URL on my OpenHAB Server using a self-singed certificate. (I cannot setup Let's Encrypt, since the webserver is on purpose not reacheable from the outside. Although this is my internal Network I still want to rely on encrypted connections.) Currently my workaround was to configure the reverse proxy such that only the webview URL is being served also with HTTP - while everything else is only served via HTTPS. But actually, this workaround is really not nice... HTTPS-WebView-URLs to external Websites with "real" certificates are working as well as non-encrypted HTTP-WebView-URLs to my internal server. I tried importing the server certificate at my Android tablet via different ways, but nothing helped.

The rest seems to be working with my OpenHAB Server using a self-signed certificate - it's only the WebView that makes problems...

kleintody commented 5 years ago

My server is running with an nginx as reverse-proxy and self-signed ssl-certificate. When using that https-URL, the Sitemap is shown except from the webview pointing to the same server if it is also using that self-signed certificate based https base url. If I change over to unencrypted http, the webview works. However, the webview with https is working in a regular browser (on pc as well as android tablet) and also in the openHAB iPhone app. I imported the certificate in various ways - and in my android tablet’s browser it is now displayed as being trusted. But in the android app it is just not working. ——— EDIT: Sorry, I just noticed that I have written a similar comment already before...

kleintody commented 5 years ago

Note that when I try wirh a webview pointing to some external URL with a regular singed ceritificate (e.g. https://google.com/) it is working.

Google
Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for.
mueller-ma commented 5 years ago

I imported the certificate in various ways - and in my android tablet’s browser it is now displayed as being trusted. But in the android app it is just not working.

Should be fixed in mobile-foss-beta-debug.zip (Code: https://github.com/mueller-ma/openhab.android/tree/network-security)

mueller-ma commented 5 years ago

@kleintody Did you find time to test my apk?

Bl4d3s commented 5 years ago

@mueller-ma Hello, I had the same problem with the HabPanel Beta View (AnchorWebView). I added your changes from the branch to my own (cause of own gcm instance). It works great. Thanks

mueller-ma commented 5 years ago

Thanks for testing. I'll make a PR to get this change in the official app. By the way you can use the official app with your own openhab cloud installation, see https://github.com/openhab/openhab-android/pull/288

mueller-ma commented 5 years ago

@maniac103 I haven't found a solution for this so far. Do you think we should add a hint to webview widgets? https://github.com/mueller-ma/openhab.android/commit/12d804042a7be410fc325f4156c4fcb36cdc3a1a

maniac103 commented 5 years ago

@mueller-ma This is an interesting idea ... certainly better than not showing any indication IMHO. But the comments above make it sound like you have found a solution? So what's missing now?

mueller-ma commented 5 years ago

1187 allows the user to add a user created CA to the system. While this is a workaround it doesn't fix the actual issue: Webviews aren't using the keystore from MTM, but from the system to validate certificates.

maniac103 commented 5 years ago

Something like https://stackoverflow.com/questions/36553190/check-in-the-onreceivedsslerror-method-of-a-webviewclient-if-a-certificate-is but using the MTM keystore to validate against?

mueller-ma commented 5 years ago

That should work for webviews to the openHAB server, but not to a third party webserver like a webcam. A valid solution though.

mueller-ma commented 5 years ago

I updated #1191: Now webviews accept the certificate if it has been trusted via MTM.

Andrea-Cecchi commented 4 years ago

hi guys i have this problem but only on my dad's phone, that is an old lg g2 with android 4.4.2

mueller-ma commented 4 years ago

old lg g2 with android 4.4.2

There's already a PR for bumping the minimum supported version to Android 5: https://github.com/openhab/openhab-android/pull/1719