Closed bbbco closed 4 years ago
Hi @bbbco , Were you able to resolve this issue ? I am also facing a similar issue with record of zipkin. Please find the stack trace below
scala.MatchError: WireSend (of class com.twitter.finagle.tracing.Annotation$WireSend$) at com.twitter.finagle.zipkin.thrift.RawZipkinTracer.record(RawZipkinTracer.scala:210) at com.twitter.finagle.zipkin.thrift.SamplingTracer.record(ZipkinTracer.scala:199) at com.twitter.finagle.tracing.DefaultTracer$.record(Tracer.scala:147) at com.twitter.finagle.tracing.Trace$$anonfun$uncheckedRecord$1.apply(Trace.scala:266) at com.twitter.finagle.tracing.Trace$$anonfun$uncheckedRecord$1.apply(Trace.scala:266) at scala.collection.immutable.List.foreach(List.scala:381) at com.twitter.finagle.tracing.Trace$.uncheckedRecord(Trace.scala:266) at com.twitter.finagle.tracing.Trace$.record(Trace.scala:312) at com.twitter.finagle.tracing.AnnotatingTracingFilter.apply(TraceInitializerFilter.scala:149) at com.twitter.finagle.Filter$$anon$2.apply(Filter.scala:85) at com.twitter.finagle.ServiceProxy.apply(Service.scala:115) at com.twitter.finagle.ServiceProxy.apply(Service.scala:115) at com.twitter.finagle.Service$$anon$2.apply(Service.scala:15) at com.twitter.finagle.service.TimeoutFilter$$anonfun$apply$2.apply(TimeoutFilter.scala:111) at com.twitter.finagle.service.TimeoutFilter$$anonfun$apply$2.apply(TimeoutFilter.scala:110) at com.twitter.util.Local.let(Local.scala:131) at com.twitter.finagle.context.Context$class.let(Context.scala:176) at com.twitter.finagle.context.MarshalledContext.let(Context.scala:209) at com.twitter.finagle.service.TimeoutFilter.apply(TimeoutFilter.scala:110) at com.twitter.finagle.Filter$$anon$2.apply(Filter.scala:85) at com.twitter.finagle.service.FailureAccrualFactory$$anonfun$apply$2$$anon$2.apply(FailureAccrualFactory.scala:206) at com.twitter.finagle.Service$$anon$2.apply(Service.scala:15) at com.twitter.finagle.service.StatsFilter.apply(StatsFilter.scala:126) at com.twitter.finagle.Filter$$anon$2.apply(Filter.scala:85) at com.twitter.finagle.Service$$anon$2.apply(Service.scala:15) at com.twitter.finagle.filter.DtabStatsFilter.apply(DtabStatsFilter.scala:37) at com.twitter.finagle.Filter$$anon$2.apply(Filter.scala:85) at com.twitter.finagle.Service$$anon$2.apply(Service.scala:15) at com.twitter.finagle.tracing.ClientDestTracingFilter.apply(DestinationTracing.scala:64) at com.twitter.finagle.Filter$$anon$2.apply(Filter.scala:85)
@puneetkhanduri Is there a way to disable sending traces to zipkin? Is zipkin needed as a dependency for diffy?
@bbbco did you find a fix for it? I am facing a very similar issues. Stacktrace:
E 0730 16:32:03.078 THREAD28: Exception propagated to the root monitor! scala.MatchError: ClientRecvError(scala.MatchError: WireSend (of class com.twitter.finagle.tracing.Annotation$WireSend$)) (of class com.twitter.finagle.tracing.Annotation$ClientRecvError) at com.twitter.finagle.zipkin.thrift.RawZipkinTracer.record(RawZipkinTracer.scala:210) at com.twitter.finagle.zipkin.thrift.SamplingTracer.record(ZipkinTracer.scala:199) at com.twitter.finagle.tracing.DefaultTracer$.record(Tracer.scala:147) at com.twitter.finagle.tracing.Trace$$anonfun$uncheckedRecord$1.apply(Trace.scala:266) at com.twitter.finagle.tracing.Trace$$anonfun$uncheckedRecord$1.apply(Trace.scala:266) at scala.collection.immutable.List.foreach(List.scala:381) at com.twitter.finagle.tracing.Trace$.uncheckedRecord(Trace.scala:266) at com.twitter.finagle.tracing.Trace$.record(Trace.scala:312) at com.twitter.finagle.tracing.AnnotatingTracingFilter$$anonfun$apply$4.apply(TraceInitializerFilter.scala:154) at com.twitter.finagle.tracing.AnnotatingTracingFilter$$anonfun$apply$4.apply(TraceInitializerFilter.scala:150) at com.twitter.util.Promise$Monitored.apply(Promise.scala:73) at com.twitter.util.Promise$Monitored.apply(Promise.scala:64) at com.twitter.util.Promise$$anon$2.run(Promise.scala:358) at com.twitter.concurrent.LocalScheduler$Activation.run(Scheduler.scala:193) at com.twitter.concurrent.LocalScheduler$Activation.submit(Scheduler.scala:140) at com.twitter.concurrent.LocalScheduler.submit(Scheduler.scala:222) at com.twitter.concurrent.Scheduler$.submit(Scheduler.scala:84) at com.twitter.util.Promise.runq(Promise.scala:342) at com.twitter.util.Promise.updateIfEmpty(Promise.scala:713) at com.twitter.util.Promise.update(Promise.scala:686) at com.twitter.util.Promise.setValue(Promise.scala:662) at com.twitter.concurrent.AsyncQueue.offer(AsyncQueue.scala:88) at com.twitter.finagle.netty3.transport.ChannelTransport.handleUpstream(ChannelTransport.scala:51) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.SimpleChannelHandler.messageReceived(SimpleChannelHandler.java:142) at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.SimpleChannelHandler.messageReceived(SimpleChannelHandler.java:142) at com.twitter.finagle.netty3.channel.ChannelRequestStatsHandler.messageReceived(ChannelRequestStatsHandler.scala:35) at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:145) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124) at com.twitter.finagle.http.codec.RespondToExpectContinue.messageReceived(RespondToExpectContinue.scala:29) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.handler.codec.http.HttpContentEncoder.messageReceived(HttpContentEncoder.java:82) at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459) at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536) at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.jboss.netty.handler.codec.http.HttpServerCodec.handleUpstream(HttpServerCodec.java:56) at com.twitter.finagle.http.SafeHttpServerCodec.handleUpstream(Codec.scala:41) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) at org.jboss.netty.channel.SimpleChannelHandler.messageReceived(SimpleChannelHandler.java:142) at com.twitter.finagle.netty3.channel.ChannelStatsHandler.messageReceived(ChannelStatsHandler.scala:80) at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108) at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337) at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
We're getting the following stack traces after running a script with many curl requests:
Maybe we are straining the process too hard? Do you have any input on the expected scalability and performance that we should expect when running multiple requests?