Handle the hostname header for the SSL verification

[jhoukem]

Describe the bug I have a case where the computer on which my program run does not have a dns server. As a work around I use my server ip address to connect to my api. This lead to an SSL certificate error since the ip address do not match the certificate hostname:
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
On Postman I can reproduce the same issue but when I manually add the Host param in the header and make it match my server hostname it work properly. When I do the same on Unirest the ip address is still used to check the ssl certificate.

To Reproduce Make an https request to a server using its real ip in the url.

Expected behavior When using an ip address in the url and the server hostname in the header under the "Host" var, the ssl verification should succeed.

If this is not a bug how would you advise me to proceed ?

[ryber]

I'll take a look but I'm not sure there is much that can be done here as the SSL validation is done down in Java and Unirest doesn't have a ton of control over it. It feels like what you are asking to do might be exploitable. In the mean time, you can disable SSL for local non-production purposes with Unirest.config().verifySsl(false);

[jhoukem]

Thank you. From what I know it is possible to do so with apache if we use a specific HttpHost constructor:
HttpHost(final InetAddress address, final String hostname, final int port, final String scheme)
but in the unirest code running the one used is: HttpHost(final String hostname, final int port, final String scheme) it is created in the CloseableHttpClient class in the determineTarget method.

From my understanding there is not much we can do about that but maybe I'm wrong...

Edit: I cannot use the verifySsl(false) config because it is code used in production. What I did instead was to configure a custom apache HttpClient that accept self signed certificate with a custom hostnameVerifier that ensure the hostname is mine. But I don't know how secure it is to be honest:

 SSLContext sslcontext = null;
            try {
                sslcontext = SSLContexts.custom()
                        .loadTrustMaterial(null, new TrustSelfSignedStrategy())
                        .build();
                SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslcontext, (s, sslSession) -> {
                    return s.equals(myHostName);// Ensure that the self signed  certificate hostname match my website hostname.
                });
                CloseableHttpClient httpclient = HttpClients.custom()
                        .setSSLSocketFactory(sslsf)
                        .build();
                config.httpClient(ApacheClient.builder(httpclient).apply(config));// Apply my previous config
            } catch (NoSuchAlgorithmException | KeyManagementException | KeyStoreException e) {
                e.printStackTrace();
            }

Do you think this is a viable option ?

[ryber]

Let me better understand what you are trying to do. When you say

As a work around I use my server ip address to connect to my api

Is this all local, as in, you have this application, which has a built in certificate running locally on your computer, but because it's local you are connecting to your IP address rather than a hostname?

and in addition, these certificates are self-signed?

I'm not sure you need to build an entire CloseableHttpClient as both the SSlContext and HostNameVerifier are exposed in the config:

SSLContext context = SSLContexts.custom()
                .loadTrustMaterial(null, new TrustSelfSignedStrategy())
                .build();
        HostnameVerifier verifier = (hostname, session) -> hostname.equals("www.zombo.com");

        Unirest.config().sslContext(context).hostnameVerifier(verifier);

Now, I wouldn't necessarily say this is something that should be in production code, but I also wouldn't host a production website with a self signed cert. For developer purposes its ok

[jhoukem]

Is this all local, as in, you have this application, which has a built in certificate running locally on your computer, but because it's local you are connecting to your IP address rather than a hostname?

No the certificate is a basic Lets'encrypt one.

I use my ip address for the request because the machine on which my program run on has no dns configured (my server hostname cannot be resolved).

This allow me to reach my webservice without having a dns (since it basically skip the resolution process by using the raw ip) but the counterpart is that now the SSL verification fails because it check for the hostname which is my raw ip address in this case (and the certificate specify the webservice hostname).

To try to fix this issue I added the header "host" to my request with my webservice hostname hoping that it will be used to check the SSL certificate (as it is the case in postman) but it turned out to be completely ignored by the implementation.

Furthermore when looking at the apache source code I see that this constructorHttpHost(final InetAddress address, final String hostname, final int port, final String scheme) would allow me to do what I want which is: to use my raw ip for the connection (InetAdress address) and use my hostname for the ssl verification (String hostname).

[ryber]

So, this is a little funky, the "Hosts" header is not really designed for use by the clients as part of the negotiation. It's intent is to allow a server to support multiple host names on a single IP: (https://tools.ietf.org/html/rfc7230#section-5.4)

We can't construct the HttpHost without both the IP address and the Hostname, There is also no guarantee that the hostname in the original request URL is in fact a IP address. I've got a little hack that does this anyway and seems to work with certificates that are still publicly resolvable:

https://github.com/Kong/unirest-java/commit/1f4cb901729125375a84aac3a57a2e6ee8f3606f#diff-22e73d69e010e7a1fecdc07df7213267R81-R100

However, I want to note that Apache Http Client won't do this hack on it's own. I'm interested in if other clients (Feign, The JDK11 Http Client, etc) will do this on their own. Postman doing it is interesting.

What I would be interested in is if the cert actually works when this is done if the domain is not resolvable via DNS. I don't have a way to test this scenario. Have you attempted to use the Apache client directly in this case and see if it worked?

[ryber]

Just for documentation curl doesn't support this:

curl -i -H "Host: sha512.badssl.com" https://104.154.89.105/
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

[jhoukem]

Yes indeed it seems that it is not supported by most clients out there. FYI I already tried with the apache client directly and it does work. Your commit seems do to exactly what I need! It is awesome! Thank you for your time. What is the next step now? Will it be merged or should I build it on my own?

[ryber]

I had to mull on it a bit. It's not exactly expected behavior, but I've also always seen postman as a closer relative to Unirest than Apache. I'll go ahead and merge it with some additional edge testing and get a release out this week.

[jhoukem]

Awesome thank you for that! 👍

[ryber]

released in 3.11.00