search

EventStore / EventStore.JVM

Event Store JVM Client
https://geteventstore.com
BSD 3-Clause "New" or "Revised" License
223 stars 40 forks source link

Eventstore connection dies and becomes unresponsive after intense read/writes #56

Open kristian-elder opened 8 years ago

kristian-elder commented 8 years ago

I'm running an import job on EventStore using the Java client, which consists of 10's of thousands of individual transactictions, each writing to a new stream.

Everything is going well and my streams get created - up to a point. After a number of writes (1000+ transactions), I'm seeing the stracktrace below in the logs, with future writes resulting in an AskTimeoutException.

Having had a look at the code, I cannot see any possible way in which this should happen. However, with this change: https://github.com/Elder-HQ/EventStore.JVM/commit/71fc93c4ff6840e681d15232f90c85c30a45d840 that avoids the pattern match, I no longer see the problem.

Stracktrace when error happens (pre fix):

15:47:35.618 [default-akka.actor.default-dispatcher-11] ERROR akka.actor.OneForOneStrategy - Expected.NoStream (of class eventstore.ExpectedVersion$NoStream$) scala.MatchError: Expected.NoStream (of class eventstore.ExpectedVersion$NoStream$) at eventstore.tcp.EventStoreProtoFormats$class.eventstore$tcp$EventStoreProtoFormats$$expectedVersion(EventStoreProtoFormats.scala:450) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.tcp.EventStoreProtoFormats$WriteEventsWriter$.toProto(EventStoreProtoFormats.scala:147) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.tcp.EventStoreProtoFormats$WriteEventsWriter$.toProto(EventStoreProtoFormats.scala:143) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.proto.DefaultProtoFormats$ProtoWriter$class.write(DefaultProtoFormats.scala:42) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.tcp.EventStoreProtoFormats$WriteEventsWriter$.write(EventStoreProtoFormats.scala:143) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.tcp.MarkerByte$.eventstore$tcp$MarkerByte$$writeMessage$1(MarkerByte.scala:80) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.tcp.MarkerByte$$anonfun$writer$2.apply(MarkerByte.scala:82) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.tcp.MarkerByte$$anonfun$writer$2.apply(MarkerByte.scala:82) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.tcp.EventStoreFormats$PackOutOutWriter$.write(EventStoreFormats.scala:66) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.tcp.EventStoreFormats$PackOutOutWriter$.write(EventStoreFormats.scala:58) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.util.BytesWriter$class.toByteString(BytesFormat.scala:18) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.tcp.EventStoreFormats$PackOutOutWriter$.toByteString(EventStoreFormats.scala:58) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.tcp.MessageByteStringAdapter$$anon$1$$anonfun$1.apply(MessageByteStringAdapter.scala:12) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.tcp.MessageByteStringAdapter$$anon$1$$anonfun$1.apply(MessageByteStringAdapter.scala:12) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.pipeline.PipelineStage$$anon$7$$anon$8$$anonfun$13.apply(Pipelines.scala:678) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.pipeline.PipelineStage$$anon$7$$anon$8$$anonfun$13.apply(Pipelines.scala:678) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.pipeline.PipelineStage$$anon$7$$anon$8$$anonfun$13.apply(Pipelines.scala:678) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.pipeline.PipelineStage$$anon$7$$anon$8$$anonfun$13.apply(Pipelines.scala:678) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.pipeline.PipelineStage$$anon$7$$anon$8$$anonfun$13.apply(Pipelines.scala:678) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.pipeline.PipelineStage$$anon$7$$anon$8$$anonfun$13.apply(Pipelines.scala:678) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.pipeline.PipelineFactory$$anon$6$$anonfun$11.apply(Pipelines.scala:323) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.pipeline.PipelineFactory$$anon$6$$anonfun$11.apply(Pipelines.scala:323) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at scala.util.Try$.apply(Try.scala:192) ~[scala-library-2.11.8.jar:na] at eventstore.pipeline.PipelineFactory$$anon$6.injectCommand(Pipelines.scala:323) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at eventstore.pipeline.TcpPipelineHandler$$anonfun$receive$1.applyOrElse(TcpPipelineHandler.scala:151) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at akka.actor.Actor$class.aroundReceive(Actor.scala:482) ~[akka-actor_2.11-2.4.6.jar:na] at eventstore.pipeline.TcpPipelineHandler.aroundReceive(TcpPipelineHandler.scala:121) ~[eventstore-client_2.11-2.2.2.jar:2.2.2] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526) [akka-actor_2.11-2.4.6.jar:na] at akka.actor.ActorCell.invoke(ActorCell.scala:495) [akka-actor_2.11-2.4.6.jar:na] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) [akka-actor_2.11-2.4.6.jar:na] at akka.dispatch.Mailbox.run(Mailbox.scala:224) [akka-actor_2.11-2.4.6.jar:na] at akka.dispatch.Mailbox.exec(Mailbox.scala:234) [akka-actor_2.11-2.4.6.jar:na] at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [scala-library-2.11.8.jar:na] at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [scala-library-2.11.8.jar:na] at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [scala-library-2.11.8.jar:na] at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [scala-library-2.11.8.jar:na]

t3hnar commented 8 years ago

Looks like I was able to reproduce the issue! Trying to fix it now.

t3hnar commented 7 years ago

@kristian-elder could you try the latest version of 2.4.0 ?