JKS not found and java.io.FileNotFoundException: https://discovery.meethue.com/

[ilker-aktuna]

I am trying to run an example app. But it throws the following exception when I use it as written in README

Future<List<HueBridge>> bridgesFuture = new HueBridgeDiscoveryService()
                        .discoverBridges(bridge -> Log.d("000000","Bridge found: " + bridge));
                final List<HueBridge> bridges;
                try {
                    bridges = bridgesFuture.get();
                } catch (ExecutionException e) {
                    throw new RuntimeException(e);
                } catch (InterruptedException e) {
                    throw new RuntimeException(e);
                }
                if( !bridges.isEmpty() ) {
                    final String bridgeIp = bridges.get(0).getIp();
                    Log.d("000000","Bridge found at " + bridgeIp);
                    // Then follow the code snippets below under the "Once you have a Bridge IP address" header
                }

E FATAL EXCEPTION: Thread-3 Process: com.example.hueconnect, PID: 6275 com.example.hueconnect.yahueapi.HueApiException: java.security.KeyStoreException: JKS not found at com.example.hueconnect.yahueapi.SecureJsonFactory.(SecureJsonFactory.java:45) at com.example.hueconnect.yahueapi.discovery.HueBridgeDiscoveryService.discoverBridges(HueBridgeDiscoveryService.java:131) at com.example.hueconnect.yahueapi.discovery.HueBridgeDiscoveryService.discoverBridges(HueBridgeDiscoveryService.java:95) at com.example.hueconnect.MainActivity$1.run(MainActivity.java:63) Caused by: java.security.KeyStoreException: JKS not found at java.security.KeyStore.getInstance(KeyStore.java:904) at com.example.hueconnect.yahueapi.SecureJsonFactory.createHueSSLContext(SecureJsonFactory.java:89) at com.example.hueconnect.yahueapi.SecureJsonFactory.(SecureJsonFactory.java:35) at com.example.hueconnect.yahueapi.discovery.HueBridgeDiscoveryService.discoverBridges(HueBridgeDiscoveryService.java:131)  at com.example.hueconnect.yahueapi.discovery.HueBridgeDiscoveryService.discoverBridges(HueBridgeDiscoveryService.java:95)  at com.example.hueconnect.MainActivity$1.run(MainActivity.java:63)  Caused by: java.security.NoSuchAlgorithmException: JKS KeyStore not available at sun.security.jca.GetInstance.getInstance(GetInstance.java:159) at java.security.Security.getImpl(Security.java:628) at java.security.KeyStore.getInstance(KeyStore.java:901) at com.example.hueconnect.yahueapi.SecureJsonFactory.createHueSSLContext(SecureJsonFactory.java:89)  at com.example.hueconnect.yahueapi.SecureJsonFactory.(SecureJsonFactory.java:35)  at com.example.hueconnect.yahueapi.discovery.HueBridgeDiscoveryService.discoverBridges(HueBridgeDiscoveryService.java:131)  at com.example.hueconnect.yahueapi.discovery.HueBridgeDiscoveryService.discoverBridges(HueBridgeDiscoveryService.java:95)  at com.example.hueconnect.MainActivity$1.run(MainActivity.java:63) 

So I changed the line in SecureJsonFactory.java:

  keystore = KeyStore.getInstance(KeyStore.getDefaultType());
  //keystore = KeyStore.getInstance("JKS");

Now, I don't get the JKS not found error:

FATAL EXCEPTION: Thread-3 Process: com.example.hueconnect, PID: 9654 java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.FileNotFoundException: https://discovery.meethue.com/ at com.example.hueconnect.MainActivity$1.run(MainActivity.java:68) Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.FileNotFoundException: https://discovery.meethue.com/ at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:372) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2049) at com.example.hueconnect.MainActivity$1.run(MainActivity.java:66) Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: https://discovery.meethue.com/ at com.example.hueconnect.yahueapi.discovery.NUPnPDiscoverer.lambda$discoverBridges$0$com-example-hueconnect-yahueapi-discovery-NUPnPDiscoverer(NUPnPDiscoverer.java:87) at com.example.hueconnect.yahueapi.discovery.NUPnPDiscoverer$$ExternalSyntheticLambda0.get(Unknown Source:6) at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1744) at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1736) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:377) at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1185) at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1658) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1625) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: java.io.FileNotFoundException: https://discovery.meethue.com/ at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:255) at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:211) at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:30) at java.net.URL.openStream(URL.java:1072) at com.fasterxml.jackson.core.TokenStreamFactory._optimizedStreamFromURL(TokenStreamFactory.java:262) at com.fasterxml.jackson.core.JsonFactory.createParser(JsonFactory.java:1115) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3714) at com.example.hueconnect.yahueapi.discovery.NUPnPDiscoverer.lambda$discoverBridges$0$com-example-hueconnect-yahueapi-discovery-NUPnPDiscoverer(NUPnPDiscoverer.java:81) at com.example.hueconnect.yahueapi.discovery.NUPnPDiscoverer$$ExternalSyntheticLambda0.get(Unknown Source:6)  at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1744)  at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1736)  at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:377)  at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1185)  at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1658)  at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1625)  at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 

what is wrong ?

[ilker-aktuna]

Then I skipped discovery and tried to connect with known IP address. I was able to get an API key

But after that when I try to connect with the assigned API key, I get this exception:

E FATAL EXCEPTION: Thread-3 Process: com.example.hueconnect, PID: 15925 com.example.hueconnect.yahueapi.HueApiException: javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. at com.example.hueconnect.yahueapi.v2.Hue.refresh(Hue.java:136) at com.example.hueconnect.yahueapi.v2.Hue.(Hue.java:104) at com.example.hueconnect.yahueapi.v2.Hue.(Hue.java:117) at com.example.hueconnect.MainActivity$1.run(MainActivity.java:99) Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. at com.android.org.conscrypt.SSLUtils.toSSLHandshakeException(SSLUtils.java:356) at com.android.org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1134) at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1089) at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:876) at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:747) at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:712) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.processDataFromSocket(ConscryptEngineSocket.java:896) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.-$$Nest$mprocessDataFromSocket(Unknown Source:0) at com.android.org.conscrypt.ConscryptEngineSocket.doHandshake(ConscryptEngineSocket.java:236) at com.android.org.conscrypt.ConscryptEngineSocket.startHandshake(ConscryptEngineSocket.java:218) at com.android.okhttp.internal.io.RealConnection.connectTls(RealConnection.java:196) at com.android.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:153) at com.android.okhttp.internal.io.RealConnection.connect(RealConnection.java:116) at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:186) at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:128) at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:97) at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:289) at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:232) at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:465) at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:411) at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:248) at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:211) at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:30) at com.example.hueconnect.yahueapi.v2.Hue.refresh(Hue.java:131) at com.example.hueconnect.yahueapi.v2.Hue.(Hue.java:104)  at com.example.hueconnect.yahueapi.v2.Hue.(Hue.java:117)  at com.example.hueconnect.MainActivity$1.run(MainActivity.java:99)  Caused by: java.security.cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. at com.android.org.conscrypt.TrustManagerImpl.verifyChain(TrustManagerImpl.java:672) at com.android.org.conscrypt.TrustManagerImpl.checkTrustedRecursive(TrustManagerImpl.java:549) at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:505) at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:425) at com.android.org.conscrypt.TrustManagerImpl.getTrustedChainForServer(TrustManagerImpl.java:353) at android.security.net.config.NetworkSecurityTrustManager.checkServerTrusted(NetworkSecurityTrustManager.java:94) at android.security.net.config.RootTrustManager.checkServerTrusted(RootTrustManager.java:90) at com.android.org.conscrypt.ConscryptEngineSocket$2.checkServerTrusted(ConscryptEngineSocket.java:163) at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:260) 2023-10-07 22:38:24.104 15925-15973 AndroidRuntime com.example.hueconnect E at com.android.org.conscrypt.ConscryptEngine.verifyCertificateChain(ConscryptEngine.java:1638) at com.android.org.conscrypt.NativeCrypto.ENGINE_SSL_read_direct(Native Method) at com.android.org.conscrypt.NativeSsl.readDirectByteBuffer(NativeSsl.java:569) at com.android.org.conscrypt.ConscryptEngine.readPlaintextDataDirect(ConscryptEngine.java:1095) at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1079) ... 24 more Caused by: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. ... 38 more

[ilker-aktuna]

is this project abandoned ?

[ilker-aktuna]

yes it is !

[ZeroOne3010]

Hi there! Sorry about this, I do not consider this project abandoned at all, the development has just been slow... Is this issue with the version from the Maven repository? Could you try with the master version from this Git repo?

[ilker-aktuna]

Hi,

This issue is with the master from Git repo. Actually I've solved the issue with connection.

But the bridge discovery does not work for me. Both MDNS and NUPNP methods in fact find an IP (I've put some debug messages to the discoverers and I can see that both methods are able to get the IP of the bridge) However, they do not return it to the call:

Future<List> bridgesFuture = new HueBridgeDiscoveryService() .discoverBridges(bridge -> Log.d("000000","Bridge found: " + bridge), HueBridgeDiscoveryService.DiscoveryMethod.MDNS);

[ilker-aktuna]

Btw, I changed "getUrlConnection" function in v2.Hue class as in following. Otherwise it was using http and did not succeed. How did it work for you ?

 URLConnection getUrlConnection(final URL url) {
    final HttpsURLConnection urlConnection;
    try {

      TrustManager[] trustAllCerts = new TrustManager[]{
              new X509TrustManager() {
                public java.security.cert.X509Certificate[] getAcceptedIssuers() {
                  Log.d("000000", "hede1 ");
                  return null;
                }
                public void checkClientTrusted(
                        java.security.cert.X509Certificate[] certs, String authType) {
                  Log.d("000000", "hede2 "+certs.toString());
                }
                public void checkServerTrusted(
                        java.security.cert.X509Certificate[] certs, String authType) throws CertificateException {
                  if(!Arrays.stream(certs).findFirst().get().getIssuerDN().toString().contains("Philips Hue")) {
                    throw new CertificateException("Certificate not valid or trusted.");
                  }
                }
              }
      };
      SSLContext sc = SSLContext.getInstance("SSL");
      sc.init(null, trustAllCerts, new java.security.SecureRandom());
      HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

      HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier() {
        @Override
        public boolean verify(String s, SSLSession sslSession) {
          String host = url.getHost();
          if(host.equals(s)) {
            return true;
          } else return false;
        }
      });

      urlConnection = (HttpsURLConnection) url.openConnection();
    } catch (IOException e) {
      throw new HueApiException(e);
    } catch (NoSuchAlgorithmException e) {
      throw new RuntimeException(e);
    } catch (KeyManagementException e) {
      throw new RuntimeException(e);
    }
    urlConnection.setRequestProperty(HUE_APPLICATION_KEY_HEADER, apiKey);
    return urlConnection;
  }

And how shall we solve the issue with discovery ? Does it work for you ?

[ZeroOne3010]

First of all, I forgot to thank you for opening this issue, so: thank you! I rely on diligent users like you to get notified if the library has any issues! 💪 Furthermore it gives me the motivation to fix it when I know somebody is using it.

I've confirmed the bridge discovery doesn't work for me either. The certificate stuff was patched elsewhere, but it seems the discovery part was overlooked! Before I manage to get it fixed, using UNVERIFIED_HTTPS protocol should get you around the issue:

new HueBridgeDiscoveryService()
        .discoverBridges(bridge -> System.out.println("Bridge found: " + bridge), HueBridgeProtocol.UNVERIFIED_HTTPS);

[ZeroOne3010]

OK I got it now actually: you (and I) are running old Bridges that are still using self-signed certificates. You can read more about it here: https://developers.meethue.com/develop/application-design-guidance/using-https/ So the only option is indeed to use the UNVERIFIED_HTTP method for now. I'll go ahead and change the default discovery protocol to UNVERIFIED_HTTPS so people don't have to worry about that by default.

[ZeroOne3010]

Would you please try the freshly minted version 3.0.0-beta-2 from the Maven repository and let me know how it works for you now! 🙏

[ilker-aktuna]

With the new version , I can confirm that the discovery is working. (I tried both MDNS and NUPNP.

I have 4 questions:

1. When I want to change the color of a zone/room or light, I use this:

                           Color a1 = new Color(0f, 1f, 0f);
                           room1.setState(new UpdateState().color(a1).brightness(100).on());

   But I get the warning that Color() is defined as private in Color/Color class. If I change it to "public" , it works. Is there any reason you set it as private ? What other way can I set color of a light ?

2. Now that you modified it to support UNVERIFIED_HTTPS , would it still work with Philips signed (non self signed) certificates ? For the sake of preserving security, would it be better to try VERIFIED HTTPS and if fails, fall back to UNVERIFIED HTTPS ?

3. Your old Hue() class has a getRaw() method and in it there is a hue.getRaw().getConfig().getBridgeId() with which I can get an ID of the Bridge. However in v2 Hue() class this is not possible. Can you add it ?

4. In v2 Hue class, there are methods getRoomByName and getZoneByName but there is no similar getLightByName method for lights. I can interpret it with something like:

                           hue.getLights().values().forEach(light -> {
                               if(light.getName().equals(dev)) {return light;}
                           });

   However, this might not be the optimal way of doing it. Why don't you add getLightByName ? is it not efficient ?

I know these questions are not related to the issue I've opened. If you prefer I can open another issue for these.

[ilker-aktuna]

Would you please try the freshly minted version 3.0.0-beta-2 from the Maven repository and let me know how it works for you now! 🙏

did you receive my message yesterday ?

[ZeroOne3010]

I did but I've been busy, sorry! Of the top of my head I can answer to your question number 1: take a look at the Color.of(...) methods. Those build the new Color instances. :) I'll get to the other questions later.

[ZeroOne3010]

2. Yes, the unverified HTTPS works with all versions. Yes, security-wise it would be better to first try the verified connection and then fall back to the unverified if it does not work. You might even want to display some notification for the user, but note that there is nothing they can do about it... So for convenience I would use the unverified connection if the application was meant for public use, but the verified one if it was for my own personal use only and my Bridge supported it.

Note that this verifiable Signify-signed certificate is actually a very recent thing, everyone has been operating for years with the unverified connection -- or most likely with the completely insecure plain HTTP connection, actually.

[ilker-aktuna]

Thanks for these answers. About color; I see that Color.of() is public but a direct access to Color() seems faster in my scenario. because I want to use a float between 0 and 1 as input. If there is no problem making it public, I will use it.

So for convenience I would use the unverified connection if the application was meant for public use, but the verified one if it was for my own personal use only and my Bridge supported it.

Can you tell me how to choose between verified and unverified one ? Btw, I believe I will have to use the "verified" one if I make an app to release on Google Play. Because Google forces using trusted certificate and not accepting all. It also forces a method to verify hostname in certificate.

[ilker-aktuna]

Hi,

will you be able to check my 3rd and 4th questions above ? And my last post ?

Btw, do you support scenes (and dynamic scenes) in the API ?

[ZeroOne3010]

3 & 4: I could add methods for these. The only issue with lights is that it's quite possible for many rooms to have lights with identical names. That possibility could be emphasized by returning a List... Of course it's possible for even the same room to have lights with the same name, but I think that's so rare that the library just picks one of them.

[ZeroOne3010]

Yes, scenes are supported:

room.getSceneByName("Tropical twilight").ifPresent(Scene::activate);

Rooms also have a getScenes() method that lists all the scenes available for that room.

[ZeroOne3010]

As for verified vs unverified HTTPS connection, for a private project I'd only use myself I'd choose the verified HTTPS if my Bridge supported it. For a public project I'd have to go with the unverified HTTPS because not everyone's Bridges have the Signify-signed certificates yet. I don't think Google Play cares whether your application talks to trusted or untrusted servers, or that it would force your app to do anything. I bet you need to sign your application somehow before submitting it to Google Play Store, but that's a different story entirely and has nothing to do with your or anyone's Hue Bridge setup.

The difference between a verified and an unverified HTTPS connection here is that if it's a verified one you can trust that your app is actually talking to a Hue Bridge, and if it's not, there could, in theory, be a man in the middle between your app and the intended Bridge. The contents are still encrypted though, so you're safe from the traditional kind of eavesdropping anyway. I find these a little far-fetched threats though. Someone would basically need to take over your LAN, and at that point you'd have bigger issues to worry about than some malicious system pretending to be your Bridge.

[ilker-aktuna]

I have more than 30 apps on Google Play. I know a lot about publishing an app :) And in fact as of this year, they care about how your app communicates with an http or https endpoint. They send you a warning if your app is using a trustmanager which accepts all certificates. And starting from december 2023 they will not approve apps which do that. see here: https://support.google.com/faqs/answer/6346016

So, I have to implement something which at least checks the publisher of the certificate (even if I will trust without authoritive signature check) (see my workaround before you made changes , post 6)

In your library, how can I select to use verified or unverified https ? is it an option to some method ? which one ?

[ZeroOne3010]

Wow, OK, well, I stand corrected! Thanks for educating me! The Hue class has a constructor where you can choose the HueBridgeProtocol you want: HTTPS or UNVERIFIED_HTTPS. The default is the unverified one if you use the constructor which does not have that parameter. You should not need your workaround if you use the verified one I think. Feel free to take a look at the code in the SecureJsonFactory class where trust managers are created.

[ZeroOne3010]

I'll work on removing the unverified part altogether and adding an as-good-as-possible verifier for the self-signed certificates.

[ilker-aktuna]

I'll work on removing the unverified part altogether and adding an as-good-as-possible verifier for the self-signed certificates.

did you have a chance to work on this ?

[ZeroOne3010]

As a matter of fact I did! I think I'm pretty close to being able to publish a new version. I would remove the old Hue class at this point and have everyone move to using the new .v2.Hue class instead, but to do that I need to refactor it some more, like by moving the new light discovery code into the new class and hopefully somehow making sure you could still add an unassigned light into a room.

[ilker-aktuna]

ok. I modified some classes slightly to be able to try HTTPS first and if it fails use UNVERIFIED_HTTPS works for me. then I added getRaw and getConfig to Hue v2 to be able to use getBridgeId with Hue v2 also works for me.

I also modified your TrustEverythingManager with checkServerTrusted as:

public void checkServerTrusted(final X509Certificate[] certs, final String authType) throws CertificateException { if(!Arrays.stream(certs).findFirst().get().getIssuerDN().toString().contains("Philips Hue")) { throw new CertificateException("Certificate not valid or trusted."); } // Do nothing }

[ZeroOne3010]

Alright, a new version is out: if you could please try 3.0.0-rc and see how that works for you! :pray:

[ilker-aktuna]

Alright, a new version is out: if you could please try 3.0.0-rc and see how that works for you! 🙏

Thanks for informing me. Actually, I am too busy nowadays to try it. As I wrote in my previous response, I made changes to solve issues I came across and it is working fine for me. If I try to move on to the new version you released, I have to adapt my app to the new code. I see in changelog that you removed old implementation.

Also, when I look at your commits today, I don't see how you handled the bridges with self signed certificate. Isn't it in the commits ?

[ZeroOne3010]

Ok, fair enough! If you do have time to try it one day, I'd appreciate if you let me know how it went.

The changes to the certificate handling are here: https://github.com/ZeroOne3010/yetanotherhueapi/commit/84ae774a6fa8ac7d41116745af1b0a00512cb709

[ilker-aktuna]

I created a new simple app which only connects to the bridge and lists rooms. No issues. So we can say it is working. At one time, I will eventually update to your new version to keep sync.