search

kamon-io / Kamon

Distributed Tracing, Metrics and Context Propagation for applications running on the JVM
https://kamon.io
Other
1.41k stars 327 forks source link

Kamon losing context in a future #430

Closed osbornk closed 7 years ago

osbornk commented 7 years ago

I am making a call to Tracer.withNewContext("myTrace", "traceToken", Map.empty, true). I also have a logger that will print the trace token.

This is inside actor A. My logger successfully prints out the trace token. This then gets forwarded to actor B. Again, my logger successfully prints out the trace token. And then I create a Future. My logger no longer prints out the trace token. If I look at Tracer.currentContext, I now get a EmptyContext instead of MetricsOnlyContext.

My application is a plain SBT application. I start it with Kamon.start(), aspectjweaver 1.8.10, kamon-core 0.6.6, kamon-akka-2.4 0.6.5, kamon-log-reported 0.6.5, kamon-scala 0.6.5, and and kamon-system-metrics 0.6.5.

So, somehow as soon as it hits the Future, the context is getting lost. Any ideas? I don't see any errors either, except the EmptyContext.

Also, this is apparently only a problem in sbt. If I run it from within a docker container, it works fine. On SBT, I just do 

run

My docker container runs with the following Java param:

bashScriptExtraDefines += """addJava "-javaagent:${lib_dir}/org.aspectj.aspectjweaver-""" + aspectjVersion + """.jar""""

Nothing else special. I also verified that this worked within SBT on version 1.6.3 with

aspectj-runner:run
osbornk commented 7 years ago

It must be an issue in the new sbt-aspectj-runner. If I use the old aspectj-runner 0.1.4 with the new Kamon libs, it works fine.

ivantopo commented 7 years ago

can you post a code sample that reproduces the issue?

osbornk commented 7 years ago

Here you go:

https://github.com/osbornk/kamon-sbt-runner-bug

Run it once with

sbt
run

You will see that the stdout statements inside the Future do not have a trace token.

Then edit plugins.sbt and switch back to aspectj-runner 0.1.4. Then do:

sbt
aspectj-runner:run

You will see that ALL stdout statements have the correct trace token.

ivantopo commented 7 years ago

hey @osbornk, thanks a lot for the code to reproduce and helping to narrow down the issue :smile:, it turns out that the problem was that Scala classes were being loaded by a different classloader that wasn't applying instrumentation, I just published version 1.0.1 of the sbt-aspectj-runner with this fix included, just upgrade and let us know if you find any other issues.

osbornk commented 7 years ago

Thanks. That worked perfectly.