CouchDB requests disconnecting

[hieronx]

On some requests for a view, the connection keeps being disconnected with the following log:

2015-08-05 06:43:20 DEBUG [webapi-akka.actor.default-dispatcher-13] c.i.c.c.Client - Making a request Request(GET,http://[ip]:5984/[db]/_design/users/_view/[view]
2015-08-05 06:43:20 DEBUG [webapi-akka.actor.default-dispatcher-13] o.h.b.p.Stage - Beginning request: Request(GET,http://[ip]:5984/[db]/_design/users/_view/[view]
2015-08-05 06:43:20 DEBUG [Thread-18] o.h.b.p.Stage - Stage org.http4s.blaze.channel.nio2.ByteBufferHead sending inbound command: Disconnected
2015-08-05 06:43:20 DEBUG [Thread-18] o.h.b.p.Stage - Stage org.http4s.client.blaze.Http1ClientStage sending outbound command: Disconnect
2015-08-05 06:43:20 DEBUG [Thread-18] o.h.b.p.Stage - closeWithError(EOF)
2015-08-05 06:43:20 DEBUG [Thread-18] o.h.b.p.Stage - org.http4s.client.blaze.Http1ClientStage shutting down at Wed Aug 05 06:43:20 UTC 2015
2015-08-05 06:43:20 DEBUG [Thread-18] o.h.b.p.Stage - closeWithError(EOF)
2015-08-05 06:43:20 DEBUG [pool-1-thread-1] o.h.b.p.Stage - Beginning request: Request(GET,http://[ip]:5984/[db]/_design/users/_view/[view]
2015-08-05 06:43:20 DEBUG [Thread-18] o.h.b.p.Stage - Stage org.http4s.blaze.channel.nio2.ByteBufferHead sending inbound command: Disconnected
2015-08-05 06:43:20 DEBUG [Thread-18] o.h.b.p.Stage - Stage org.http4s.client.blaze.Http1ClientStage sending outbound command: Disconnect
2015-08-05 06:43:20 DEBUG [Thread-18] o.h.b.p.Stage - closeWithError(EOF)
2015-08-05 06:43:20 DEBUG [Thread-18] o.h.b.p.Stage - org.http4s.client.blaze.Http1ClientStage shutting down at Wed Aug 05 06:43:20 UTC 2015
2015-08-05 06:43:20 DEBUG [Thread-18] o.h.b.p.Stage - closeWithError(EOF)

The error seems to be the same as the following issue reported at the underlying http library: https://github.com/http4s/http4s/issues/349 (look at the "client log").

Any idea why this is going wrong in this library as well?

[beloglazov]

In your case it doesn't seem to be caused by a timeout, as in the linked issue. Does it always happen on a particular request? Would you be able to provide a simple code example to reproduce this?

[hieronx]

No it seems to be happening with various requests. However, it's hard to reproduce since it's very irregular (sometimes it just works, sometimes it times out).

Also, it might be relevant to say that they're requests to a Cloudant database.

[hieronx]

This is the (relevant) Scala code used for requesting views:

private lazy val couch = CouchDb(dbHost, dbPort, https = false, dbUser, dbPassword)

val db = {
  log.info(s"Using database $dbName @ $dbHost:$dbPort")
  couch.db(dbName, typeMapping)
}

protected def getView(designName: String, viewName: String) = {
  val view = db.query.view[String, A](designName, viewName).get
  val couchKeyVals = view.query
  val result = couchKeyVals.map(_.rows.map(kv => withId(kv.id, kv.value)).toList)
  result.toScalaFuture
}

[beloglazov]

@bryce-anderson sorry for disturbing, but do you have any ideas on why this problem might appear? Thanks!

[bryce-anderson]

Sorry for the slow reply. I can't find anything specific in those log messages. They look like a case of 'the server simply hung up'. Are there log events from the other side? I guessing like they would be buried but may be useful.

My only guess is that perhaps it has to do with using the http4s PooledClient (used here). I don't know if its possible to switch that in this case but it might be worth trying.

[ermyas]

@beloglazov have you been able to reproduce this issue? I was not able to (running repeated concurrent requests to Cloudant and couch)

[beloglazov]

@bryce-anderson thanks a lot for taking a look!

@ermyas sorry, I haven't had a chance to try it, but I believe you :)

@offerijns do you still have this issue? Have you tried doing what @bryce-anderson suggested? Could it have been just a temporary issue of Cloudant? We could also try updating to http4s 0.8.4 and swapping the http4s client to SimpleHttp1Client as a test.

[bryce-anderson]

I think this is a http4s bug after all. See https://github.com/http4s/http4s/commit/8d8c66e446ec26d4c5e8a38e8388938eea1cc72b. Basically, when the client was receiving a response with the length defined by EOF, there was a race between shutting down the stage and checking that the connection shut down normally. I'm not 100% sure its your bug, but it could be. At any rate, that commit is in v0.9.3 and v0.8.6, both of on their way to central.

[hieronx]

Sorry for the late reply, I was out of the country for a little while.

The bug you're explaining does sound like the same one I'm reporting! I'll wait for the fix to be released and try it out then—and let you know whether it works. Thanks for looking into this!

[beloglazov]

@offerijns #10 is now merged, could you please check if it fixes this issue?