Connection reset exception when calling database.findByIndex() method

[TigerAVAF6R]

We noticed the connection reset exception happened sometimes (not ALL the time) when our application is querying from cloudant database, below is some detail info:

• CloudantClient (java-cloudant) version(s) : 2.6.1
• Java version (including vendor and platform). IBM JDK 1.7
• If you're using the optional okhttp dependency. YES, using OK HTTP

• A small code sample that demonstrates the issue. Below is a part of the code. try { String query = "a sample query here"; List list = userdb.findByIndex(query, Users.class);

  } catch (CouchDbException e) {
      throw new InternalServerException(500103);
  } catch (Exception e) {
      throw new InternalServerException(500102);
  }

The exception stack is like below: com.cloudant.http.interceptors.CookieInterceptor E Error reading response code or body from request java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:220) at java.net.SocketInputStream.read(SocketInputStream.java:152) at com.ibm.jsse2.a.a(a.java:79) at com.ibm.jsse2.a.a(a.java:182) at com.ibm.jsse2.as.a(as.java:883) at com.ibm.jsse2.as.a(as.java:382) at com.ibm.jsse2.e.read(e.java:16) at okio.Okio$2.read(Okio.java:139) at okio.AsyncTimeout$2.read(AsyncTimeout.java:211) at okio.RealBufferedSource.indexOf(RealBufferedSource.java:306) at okio.RealBufferedSource.indexOf(RealBufferedSource.java:300) at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:196) at com.squareup.okhttp.internal.http.Http1xStream.readResponse(Http1xStream.java:186) at com.squareup.okhttp.internal.http.Http1xStream.readResponseHeaders(Http1xStream.java:127) at com.squareup.okhttp.internal.http.HttpEngine.readNetworkResponse(HttpEngine.java:737) at com.squareup.okhttp.internal.http.HttpEngine.readResponse(HttpEngine.java:609) at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:460) at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:399) at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:527) at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105) at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:25) at com.cloudant.http.interceptors.CookieInterceptor.interceptResponse(CookieInterceptor.java:142) at com.cloudant.http.HttpConnection.execute(HttpConnection.java:358) at com.cloudant.client.org.lightcouch.CouchDbClient.execute(CouchDbClient.java:481) at com.cloudant.client.org.lightcouch.CouchDbClient.executeToInputStream(CouchDbClient.java:572) at com.cloudant.client.api.Database.findByIndex(Database.java:377) at com.cloudant.client.api.Database.findByIndex(Database.java:333) at com.ibm.ngps.dao.UsersDao.getByIUI(UsersDao.java:211) at com.ibm.ngps.service.UserGetService.searchByParam(UserGetService.java:135) at com.ibm.ngps.resource.UsersResource.getUser(UsersResource.java:190) at sun.reflect.GeneratedMethodAccessor189.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:508) at com.ibm.ws.jaxrs20.server.LibertyJaxRsServerFactoryBean.performInvocation(LibertyJaxRsServerFactoryBean.java:651) at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.performInvocation(LibertyJaxRsInvoker.java:115) at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96) at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:210) at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:189) at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:381) at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:99) at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:61) at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:99) at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:124) at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:265) at com.ibm.ws.jaxrs20.endpoint.AbstractJaxRsWebEndpoint.invoke(AbstractJaxRsWebEndpoint.java:134) at com.ibm.websphere.jaxrs.server.IBMRestServlet.handleRequest(IBMRestServlet.java:149) at com.ibm.websphere.jaxrs.server.IBMRestServlet.doGet(IBMRestServlet.java:115) at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) at com.ibm.websphere.jaxrs.server.IBMRestServlet.service(IBMRestServlet.java:99) at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1290) at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:778) at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:475) at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1161) at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:82) at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:945) at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.run(DynamicVirtualHost.java:280) at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink$TaskWrapper.run(HttpDispatcherLink.java:967) at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.wrapHandlerAndExecute(HttpDispatcherLink.java:359) at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready(HttpDispatcherLink.java:318) at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:471) at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest(HttpInboundLink.java:405) at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest(HttpInboundLink.java:285) at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.ready(HttpInboundLink.java:256) at com.ibm.ws.channel.ssl.internal.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1043) at com.ibm.ws.channel.ssl.internal.SSLConnectionLink.readyInboundPostHandshake(SSLConnectionLink.java:709) at com.ibm.ws.channel.ssl.internal.SSLConnectionLink$MyHandshakeCompletedCallback.complete(SSLConnectionLink.java:397) at com.ibm.ws.channel.ssl.internal.SSLUtils.handleHandshake(SSLUtils.java:976) at com.ibm.ws.channel.ssl.internal.SSLHandshakeIOCallback.complete(SSLHandshakeIOCallback.java:88) at com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete(WorkQueueManager.java:504) at com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO(WorkQueueManager.java:574) at com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun(WorkQueueManager.java:929) at com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run(WorkQueueManager.java:1018) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.lang.Thread.run(Thread.java:785)

[ricellis]

As is clear from the java.net.SocketException: Connection reset in the stack trace this issue is caused by a connection reset. I believe this exception is typically due to a TCP RST flag being received from the server.

Do you have proxy servers or firewalls that might be resetting the connection?

There may be a server side issue, if you are able to record the times when it happens you could consider contacting Cloudant support and seeing if there is any issue with your server at those times.

Are you able to identify any pattern to the exceptions? For example do they occur only when querying after a period of inactivity? That would help us try to identify, for instance, if there was some kind of stale connection issue.

[ricellis]

Closing as stale, please re-open if you provide answers to the above questions.