search

cloudant / java-cloudant

A Java client for Cloudant
Apache License 2.0
79 stars 68 forks source link

IMPORTANT: Unable to connect to cloudant using Proxy #282

Closed neekrish closed 8 years ago

neekrish commented 8 years ago

When using the cloudant client to connect to cloudant.com using a proxy server, the below exception is seen. Also to add for proxy to work I had to add this code snippet too:

`
Authenticator authenticator = new Authenticator() {

                public PasswordAuthentication getPasswordAuthentication() {
                    return (new PasswordAuthentication(cloudantProxyUsername,
                            cloudantProxyPassword.toCharArray()));
                }
            };
            Authenticator.setDefault(authenticator);

`

java.net.ConnectException: Connection timed out at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.net.SocksSocketImpl.connect(Unknown Source) at java.net.Socket.connect(Unknown Source) at com.ibm.jsse2.as.connect(Unknown Source) at sun.net.NetworkClient.doConnect(Unknown Source) at sun.net.www.http.HttpClient.openServer(Unknown Source) at sun.net.www.http.HttpClient.openServer(Unknown Source) at com.ibm.net.ssl.www2.protocol.https.c.(Unknown Source) at com.ibm.net.ssl.www2.protocol.https.c.a(Unknown Source) at com.ibm.net.ssl.www2.protocol.https.d.getNewHttpClient(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source) at com.ibm.net.ssl.www2.protocol.https.d.connect(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(Unknown Source) at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(Unknown Source) at com.ibm.net.ssl.www2.protocol.https.b.getOutputStream(Unknown Source) at com.cloudant.http.HttpConnection.execute(HttpConnection.java:329) at com.cloudant.http.interceptors.CookieInterceptor.getCookie(CookieInterceptor.java:172) at com.cloudant.http.interceptors.CookieInterceptor.interceptRequest(CookieInterceptor.java:84) at com.cloudant.http.HttpConnection.execute(HttpConnection.java:306) at com.cloudant.client.org.lightcouch.CouchDbClient.execute(CouchDbClient.java:470) at com.cloudant.client.org.lightcouch.CouchDbClient.executeToInputStream(CouchDbClient.java:560) at com.cloudant.client.org.lightcouch.CouchDbClient.executeToResponse(CouchDbClient.java:282) at com.cloudant.client.org.lightcouch.CouchDbClient.createDB(CouchDbClient.java:204) at com.cloudant.client.org.lightcouch.CouchDatabaseBase.(CouchDatabaseBase.java:63) at com.cloudant.client.org.lightcouch.CouchDatabase.(CouchDatabase.java:26) at com.cloudant.client.org.lightcouch.CouchDbClient.database(CouchDbClient.java:167) at com.cloudant.client.api.CloudantClient.database(CloudantClient.java:217)

ricellis commented 8 years ago

Did you construct your CloudantClient with something like this?

ClientBuilder.account("yourCloudantAccount")
             .username("yourAPIKey")
             .password("yourAPIKeyPassphrase")
             .proxyURL(yourProxyURL)
             .proxyUser(yourProxyUser)
             .proxyPassword(yourProxyPassword)
             .build();
neekrish commented 8 years ago

Yes ... That's right... Like this:

Authenticator authenticator = new Authenticator() {

                public PasswordAuthentication getPasswordAuthentication() {
                    return (new PasswordAuthentication(cloudantProxyUsername,
                            cloudantProxyPassword.toCharArray()));
                }
            };
            Authenticator.setDefault(authenticator);

builder.proxyPassword(cloudantProxyPassword).proxyURL(new URL(cloudantProxyUrl)).proxyUser(cloudantProxyUsername); client = builder.build();

ricellis commented 8 years ago

What exception did you get before you added the Authenticator? I would not expect an Authenticator to be necessary - using the proxyUser and proxyPassword instructs the client to add the Proxy-Authorization header to the requests using basic auth.

Anyway I wouldn't expect to see that stack trace for an auth failure (should get a 401/407 response). The connection timeout usually suggests that the client couldn't reach the necessary address. Is your proxy running on a non-standard port and have you specified that correctly in the URL?

neekrish commented 8 years ago

Before adding authenticator I got this exception: com.cloudant.client.org.lightcouch.CouchDbException: 407 Proxy Authentication Required    at com.cloudant.client.org.lightcouch.CouchDbClient.execute(CouchDbClient.java:513)     at com.cloudant.client.org.lightcouch.CouchDbClient.executeToInputStream(CouchDbClient.java:560)     at com.cloudant.client.org.lightcouch.CouchDbClient.executeToResponse(CouchDbClient.java:282)     at com.cloudant.client.org.lightcouch.CouchDbClient.createDB(CouchDbClient.java:204)     at com.cloudant.client.org.lightcouch.CouchDatabaseBase.(CouchDatabaseBase.java:63)     at com.cloudant.client.org.lightcouch.CouchDatabase.(CouchDatabase.java:26)     at com.cloudant.client.org.lightcouch.CouchDbClient.database(CouchDbClient.java:167)     at com.cloudant.client.api.CloudantClient.database(CloudantClient.java:217)

The node client for cloudant is able to connect to it ... Its only the java cloudant client that's not working.

The proxy is running on 1081 port... 407 exception indicates that the authentication is not happening .. Hence adding the Authenticator helped in getting past this issue. But it hit the connection timeout issue.

EDIT: Is the username and password of proxy really used in the cloudant-client code? Can you point me to the code that uses the username and password, so that I can debug?

ricellis commented 8 years ago

https://github.com/cloudant/java-cloudant/blob/master/cloudant-client/src/main/java/com/cloudant/client/api/ClientBuilder.java#L236 adds the ProxyAuthInterceptor if the proxyUser is not null.

neekrish commented 8 years ago

Thanks ... I'll investigate further and update you tomorrow ... I'll not use the Authenticator code.

neekrish commented 8 years ago

After a long investigation today ... Here's the only option that worked

Note that http.proxyHost didn't work too. Also setting the "-Dhttps.proxyHost" and using the "Proxy-Authorization" didn't work.

I would prefer that you document this and close this issue.

ricellis commented 8 years ago

I'm glad you found something that works for you. Using those system properties with an Authenticator configures the proxy for all HttpURLConnection at the JVM level. On the other hand the ClientBuilder options configure the proxy on each HttpURLConnection in the CloudantClient and it isn't obvious to me why that is causing a difference in this case. Are you able to provide any details about the proxy server you are using? It might help us test and iron out any issues in that area.

ricellis commented 8 years ago

OK so having investigated this a bit more and written some more tests I think the problem is that our Proxy-Authorization header is not added to the initial HTTP CONNECT request when we need to create an SSL tunnel through the proxy to an https server. The header is present on the client requests, but those are, of course, encrypted and not visible to the proxy itself.

I'm not sure if there is anything we can do to the HttpURLConnection to make it present the credentials on a per client basis (as opposed to the global authenticator), but at the very least we should improve the documentation in this regard.

neekrish commented 8 years ago

Thanks a lot for focus on this ... I did lot more tests with the approach I took and found one more related issue:

Is there a build with the pull request of this defect that I can use to test?

ricellis commented 8 years ago

@neekrish I've merged my changes, so you could test with a version 2.5.2-SNAPSHOT build from the snapshot repository at https://oss.sonatype.org/content/repositories/snapshots - that repository also has the updated javadoc jar.

After this change with your use case you should no longer need the https.proxy* properties, you should be able to use builder.proxyURL(new URL("http://...")). You will still need to set the default Authenticator to provide the credentials.

I'm not sure this is going to help with the new issue you describe though. Typically I would not expect a Connection timed out except when a host isn't reachable. Although since before you saw a Connection timed out when the auth wasn't correct Is it possible that something else (another thread or program) is changing the default Authenticator and therefore removing your credentials?

Am I correct in understanding that you are needing to restart the proxy server? Perhaps it is running out of resources like connections and therefore not able to accept any more requests.

ricellis commented 8 years ago

I released the proxy changes I made as part of 2.6.0. I'm going to close this issue, please open another issue if there is another problem with talking through a proxy using this library. Thanks.