search

kamon-io / Kamon

Distributed Tracing, Metrics and Context Propagation for applications running on the JVM
https://kamon.io
Other
1.4k stars 326 forks source link

Getting Exception related to tags #766

Open richiesgr opened 4 years ago

richiesgr commented 4 years ago

Using the version 2.1.0 I get a lot of these exception java.lang.ArrayIndexOutOfBoundsException: Index 17 out of bounds for length 17 at kamon.tag.TagSet$Builder$ChainedArray.addPair(TagSet.scala:520) at kamon.tag.TagSet$Builder$ChainedArray.$anonfun$add$1(TagSet.scala:473) at kamon.tag.TagSet$Builder$ChainedArray.$anonfun$add$1$adapted(TagSet.scala:473) at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:553) at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:551) at kamon.tag.TagSet$$anon$4.foreach(TagSet.scala:177) at kamon.tag.TagSet$Builder$ChainedArray.add(TagSet.scala:473) at kamon.trace.Tracer$MutableSpanBuilder.tag(Tracer.scala:176) at com.dv.phoenix.eventgateway.HttpServer.$anonfun$asyncRequestHandler$1(HttpServer.scala:64) at kamon.instrumentation.akka.http.Http2BlueprintInterceptor$HandlerWithEndpoint.apply(AkkaHttpServerInstrumentation.scala:339) at kamon.instrumentation.akka.http.Http2BlueprintInterceptor$HandlerWithEndpoint.apply(AkkaHttpServerInstrumentation.scala:336) at akka.http.scaladsl.Http2Ext.$anonfun$handleUpgradeRequests$1(Http2.scala:144) at akka.stream.impl.fusing.MapAsync$$anon$30.onPush(Ops.scala:1289) at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:541) at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:423) at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:624) at akka.stream.impl.fusing.ActorGraphInterpreter$SimpleBoundaryEvent.execute(ActorGraphInterpreter.scala:55) at akka.stream.impl.fusing.ActorGraphInterpreter$SimpleBoundaryEvent.execute$(ActorGraphInterpreter.scala:51) at akka.stream.impl.fusing.ActorGraphInterpreter$BatchingActorInputBoundary$OnNext.execute(ActorGraphInterpreter.scala:94) at akka.stream.impl.fusing.GraphInterpreterShell.processEvent(ActorGraphInterpreter.scala:599) at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:768) at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:783) at akka.actor.Actor.aroundReceive(Actor.scala:534) at akka.actor.Actor.aroundReceive$(Actor.scala:532) at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:690) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:573) at akka.actor.ActorCell.invoke(ActorCell.scala:543) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:269) at akka.dispatch.Mailbox.run(Mailbox.scala:230) at kamon.instrumentation.executor.ExecutorInstrumentation$InstrumentedForkJoinPool$TimingRunnable.run(ExecutorInstrumentation.scala:662) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:47) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1016) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1665) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1598) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)

alexandru commented 3 years ago

Hello,

We are also getting this exception with Kamon version 2.1.9.

SimunKaracic commented 3 years ago

What framework are you using Kamon with (I'm guessing akka http)? What scala/java version?

alexandru commented 3 years ago

@SimunKaracic issue was reproduced with:

Kamon configuration in application.conf:

kamon.prometheus {
  include-environment-tags = true
  embedded-server {
    hostname = 0.0.0.0
    port = 9095
  }
}
SimunKaracic commented 3 years ago

Ah, we didn't have support for akka-http 10.2.x back then. Please upgrade to version 2.1.12 (earliest with Akka-HTTP 10.2 support).

It's updated to use 10.2.3, but if the error keeps happening after the upgrade, we'll instrument 10.2.4 as well

ivantopo commented 3 years ago

@alexandru are you seeing this issue even after the upgrade? Just looking at the stack trace makes me think the problem could be somewhere else, not specifically related to the Akka HTTP version. Please let us know!

alexandru commented 2 years ago

Our issue is still reproducing:

Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 18 out of bounds for length 17
    at kamon.tag.TagSet$Builder$ChainedArray.addPair(TagSet.scala:519)
    at kamon.tag.TagSet$Builder$ChainedArray.add(TagSet.scala:467)
    at kamon.trace.Span$Local.createMetricTags(Span.scala:662)
    at kamon.trace.Span$Local.finish(Span.scala:608)
    at kamon.trace.Span$Local.finish(Span.scala:587)
ivantopo commented 2 years ago

Hey @alexandru,

I have been trying to reproduce this issue with no success. Can you please share any info that might help reproduce it? Things that come to mind are: