kamon-io / Kamon

Distributed Tracing, Metrics and Context Propagation for applications running on the JVM
https://kamon.io
Other
1.41k stars 327 forks source link

Context propagation is broken for Promises #1355

Open qwe2 opened 1 month ago

qwe2 commented 1 month ago

The following code prints null on Kamon v2.7.3 / Kanela v1.0.18 (scala 2.13.14):

package foo

import kamon.Kamon
import kamon.tag.{Lookups, TagSet}

import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.DurationInt
import scala.concurrent.{Await, Promise}
import scala.util.Success

object Foo {
  def main(args: Array[String]): Unit = {
    Kamon.init()

    val prom = Promise[Int]()

    val context = Kamon.currentContext()
    val ts = TagSet.builder()
    ts.add("foo", "bar")

    val newCtx = context.withTags(ts.build())

    Kamon.runWithContext(newCtx) {
      prom.complete(Success(123))
    }

    val res = prom.future.map { _ =>
      val context = Kamon.currentContext()
      println(context.getTag(Lookups.plain("foo"))) // <---- should print "bar"
    }

    Await.result(res, 10.seconds)
  }
}

This should print bar, which it does on older versions. Tested on 2.1.3 where it was still working. I suspect the issue is that FutureChainingInstrumentation has been deprecated and disabled by default, and the replacement does not work on Promise. The above code works correctly on Kamon v2.7.3 with the below config:

kanela {
    modules {
        executor-service.enabled = false
        scala-future.enabled = true
    }
}

Note that unless executor-service is disabled, enabling scala-future results in

[info] Initializing Kamon Telemetry v2.7.3 / Kanela v1.0.18
[error] Exception in thread "main" java.lang.ClassFormatError: Duplicate interface name "kamon/instrumentation/context/HasContext" in class file scala/concurrent/impl/CallbackRunnable
[error]         at java.base/java.lang.ClassLoader.defineClass1(Native Method)
[error]         at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022)
[error]         at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
[error]         at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
[error]         at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)
[error]         at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
[error]         at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
[error]         at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
[error]         at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
[error]         at scala.concurrent.impl.Promise$DefaultPromise.onComplete(Promise.scala:317)
[error]         at scala.concurrent.impl.Promise.transform(Promise.scala:41)
[error]         at scala.concurrent.impl.Promise.transform$(Promise.scala:39)
[error]         at scala.concurrent.impl.Promise$DefaultPromise.transform(Promise.scala:197)
[error]         at scala.concurrent.Future.map(Future.scala:292)
[error]         at scala.concurrent.Future.map$(Future.scala:292)
[error]         at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:197)
[error]         at foo.Foo$.main(Foo.scala:27)
[error]         at foo.Foo.main(Foo.scala)

~Additionally, the scala-future module is completely broken when used together with sbt-kanela-runner 2.1.0. For some reason, scala.util.Success does never get instrumented (interestingly, Failure does) when running on this version. It is working properly on sbt-kanela-runner 2.0.14 however. I have not investigated this any further.~ This was an issue with javaagent missing from javaOptions (https://github.com/kamon-io/sbt-kanela-runner/issues/30).

qwe2 commented 1 month ago

This issue breaks context propagation in Play applications when using filters, since filters use Promise: https://github.com/playframework/playframework/blob/d1f1b9f8cd8d564933381967b4e623b8842c2377/core/play/src/main/scala/play/api/mvc/Filters.scala#L62

Using the following config seems to work as a workaround for Play filters (taken from the docs as better option to outright disabling executor-service):

kanela {
    modules {
        executor-service {
            exclude += "scala.concurrent.impl.*"
        }
        scala-future.enabled = true
    }
}