search

puniverse / quasar

Fibers, Channels and Actors for the JVM
http://docs.paralleluniverse.co/quasar/
Other
4.56k stars 575 forks source link

Quasar fails to instrument my usage of Kotlin lambdas #142

Open Jire opened 8 years ago

Jire commented 8 years ago

Quasar fails to instrument my usage of Kotlin lambdas.

The following code https://github.com/Jire/Abendigo/blob/master/src/main/kotlin/org/abendigo/plugin/csgo/RadarPlugin.kt

Produces


java.lang.NoClassDefFoundError: Could not initialize class org.abendigo.csgo.CSGOKt
    at org.abendigo.plugin.csgo.RadarPlugin$enable$1.invoke(RadarPlugin.kt:13)
    at org.abendigo.plugin.csgo.RadarPlugin$enable$1.invoke(RadarPlugin.kt:8)
    at org.abendigo.AbendigoKt$every$1.invoke(Abendigo.kt:22)
    at org.abendigo.AbendigoKt$every$1.invoke(Abendigo.kt)
    at co.paralleluniverse.kotlin.KotlinKt$fiber$sc$1.run(Kotlin.kt:29)
    at co.paralleluniverse.fibers.Fiber.run(Fiber.java:1024)
    at co.paralleluniverse.fibers.Fiber.run1(Fiber.java:1019)
    at co.paralleluniverse.fibers.Fiber.exec(Fiber.java:730)
    at co.paralleluniverse.fibers.FiberForkJoinScheduler$FiberForkJoinTask.exec1(FiberForkJoinScheduler.java:265)
    at co.paralleluniverse.concurrent.forkjoin.ParkableForkJoinTask.doExec(ParkableForkJoinTask.java:116)
    at co.paralleluniverse.concurrent.forkjoin.ParkableForkJoinTask.exec(ParkableForkJoinTask.java:73)
    at jsr166e.ForkJoinTask.doExec(ForkJoinTask.java:261)
    at jsr166e.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:988)
    at jsr166e.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
    at jsr166e.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

using Kotlin Beta 3 official version with Quasar 0.7.4 agent. My application will not run with AOT. (Massive reports of uninstrumented cases, as seen by this bug: https://github.com/puniverse/quasar/issues/140)

The reason I am try/catching is because otherwise I receive VerifyError. You can view the entire project from here: https://github.com/Jire/Abendigo/tree/master/src/main/kotlin/org/abendigo

Jire commented 8 years ago

May be related to using a Lock. https://github.com/Jire/kotmem/blob/master/src/main/kotlin/org/jire/kotmem/Kotmem.kt

circlespainter commented 8 years ago

Instead of using java.util.concurrent you should use co.paralleluniverse.strands.concurrent; it's basically the same API but it works both with threads and fibers. There are also some useful Kotlin bindings in co.paralleluniverse.kotlin.concurrent.

Jire commented 8 years ago

@circlespainter I made the switch, unfortunately still the same problem. :-(

I tried annotating everything with @Suspendable, tried using only one or the other of my two plugin classes, tried using the -Dco.paralleluniverse.fibers.verifyInstrumentation=true, tried removing code from my CSGO.kt file, tried changing the names of the file and code inside CSGO.kt... it doesn't make sense why with regular threads everything works fine but with fibers it repeatedly throws a no class def:

java.lang.NoClassDefFoundError: Could not initialize class org.abendigo.csgo.CSGOKt
    at org.abendigo.plugin.csgo.BunnyHopPlugin$enable$1.invoke(BunnyHopPlugin.kt:18)
    at org.abendigo.plugin.csgo.BunnyHopPlugin$enable$1.invoke(BunnyHopPlugin.kt:12)
    at org.abendigo.AbendigoKt$every$1.invoke(Abendigo.kt:22)
    at org.abendigo.AbendigoKt$every$1.invoke(Abendigo.kt)
    at co.paralleluniverse.kotlin.KotlinKt$fiber$sc$1.run(Kotlin.kt:29)
    at co.paralleluniverse.fibers.Fiber.run(Fiber.java:1024)
    at co.paralleluniverse.fibers.Fiber.run1(Fiber.java:1019)
    at co.paralleluniverse.fibers.Fiber.exec(Fiber.java:730)
    at co.paralleluniverse.fibers.FiberForkJoinScheduler$FiberForkJoinTask.exec1(FiberForkJoinScheduler.java:265)
    at co.paralleluniverse.concurrent.forkjoin.ParkableForkJoinTask.doExec(ParkableForkJoinTask.java:116)
    at co.paralleluniverse.concurrent.forkjoin.ParkableForkJoinTask.exec(ParkableForkJoinTask.java:73)
    at jsr166e.ForkJoinTask.doExec(ForkJoinTask.java:261)
    at jsr166e.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:988)
    at jsr166e.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
    at jsr166e.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Jire commented 8 years ago

Something interesting I tried was using regular threads without any mention of Quasar but I enabled instrumentation through the agent and the error above was thrown. Removing the agent flag results in the code working.

circlespainter commented 8 years ago

Have you tried enabling instrumentation verbosity/debugging/checking by appending =vdc to the agent to see if the instrumentation logs mention some problem about that class?

Jire commented 8 years ago

I get these warnings although they are not constant (as in they only happen once and then no longer happen). This I believe is because my library is initialized lazily, and the first time it does so it takes a few hundred millis to get a response.

WARNING: fiber Fiber@10000001:fiber-10000001[task: ParkableForkJoinTask@734e3400(Fiber@10000001), target: co.paralleluniverse.kotlin.KotlinKt$fiber$sc$1@1904af69, scheduler: co.paralleluniverse.fibers.FiberForkJoinScheduler@7e388c1f] is hogging the CPU or blocking a thread.
    at java.net.NetworkInterface.getAll(Native Method)
    at java.net.NetworkInterface.getNetworkInterfaces(NetworkInterface.java:343)
    at sun.security.provider.SeedGenerator.addNetworkAdapterInfo(SeedGenerator.java:233)
    at sun.security.provider.SeedGenerator.access$000(SeedGenerator.java:80)
    at sun.security.provider.SeedGenerator$1.run(SeedGenerator.java:183)
    at sun.security.provider.SeedGenerator$1.run(SeedGenerator.java:168)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.provider.SeedGenerator.getSystemEntropy(SeedGenerator.java:168)
    at sun.security.provider.SecureRandom$SeederHolder.<clinit>(SecureRandom.java:201)
    at sun.security.provider.SecureRandom.engineNextBytes(SecureRandom.java:221)
    at java.security.SecureRandom.nextBytes(SecureRandom.java:468)
    at java.security.SecureRandom.next(SecureRandom.java:491)
    at java.util.Random.nextLong(Random.java:424)
    at java.io.File$TempDirectory.generateFile(File.java:1905)
    at java.io.File.createTempFile(File.java:2010)
    at com.sun.jna.Native.extractFromResourcePath(Native.java:963)
    at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:841)
    at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826)
    at com.sun.jna.Native.<clinit>(Native.java:140)
    at org.jire.kotmem.unsafe.User32.<init>(User32.kt:10)
    at org.jire.kotmem.unsafe.User32.<clinit>(User32.kt:5)
    at org.jire.kotmem.keys.state(Kotmem.kt:29)
    at org.jire.kotmem.keys.get(Kotmem.kt:33)
    at org.abendigo.plugin.csgo.BunnyHopPlugin$enable$1.invoke(BunnyHopPlugin.kt:16)
    at org.abendigo.plugin.csgo.BunnyHopPlugin$enable$1.invoke(BunnyHopPlugin.kt:11)
    at org.abendigo.AbendigoKt$every$1.invoke(Abendigo.kt:20)
    at org.abendigo.AbendigoKt$every$1.invoke(Abendigo.kt)
    at co.paralleluniverse.kotlin.KotlinKt$fiber$sc$1.run(Kotlin.kt:29)
    at co.paralleluniverse.fibers.Fiber.run(Fiber.java:1024)
WARNING: fiber Fiber@10000001:fiber-10000001[task: ParkableForkJoinTask@734e3400(Fiber@10000001), target: co.paralleluniverse.kotlin.KotlinKt$fiber$sc$1@1904af69, scheduler: co.paralleluniverse.fibers.FiberForkJoinScheduler@7e388c1f] is hogging the CPU or blocking a thread.
    at java.io.WinNTFileSystem.createFileExclusively(Native Method)
    at java.io.File.createTempFile(File.java:2024)
    at com.sun.jna.Native.extractFromResourcePath(Native.java:963)
    at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:841)
    at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826)
    at com.sun.jna.Native.<clinit>(Native.java:140)
    at org.jire.kotmem.unsafe.User32.<init>(User32.kt:10)
    at org.jire.kotmem.unsafe.User32.<clinit>(User32.kt:5)
    at org.jire.kotmem.keys.state(Kotmem.kt:29)
    at org.jire.kotmem.keys.get(Kotmem.kt:33)
    at org.abendigo.plugin.csgo.BunnyHopPlugin$enable$1.invoke(BunnyHopPlugin.kt:16)
    at org.abendigo.plugin.csgo.BunnyHopPlugin$enable$1.invoke(BunnyHopPlugin.kt:11)
    at org.abendigo.AbendigoKt$every$1.invoke(Abendigo.kt:20)
    at org.abendigo.AbendigoKt$every$1.invoke(Abendigo.kt)
    at co.paralleluniverse.kotlin.KotlinKt$fiber$sc$1.run(Kotlin.kt:29)
    at co.paralleluniverse.fibers.Fiber.run(Fiber.java:1024)

You can view the entirety of execution here: https://gist.github.com/Jire/fbc806256015ac004284

circlespainter commented 8 years ago

Can you share a runnable program using the project that triggers the issue?

Jire commented 8 years ago

@circlespainter The program uses JNA to do native memory interfacing on Windows. The program requires both Windows and to be in a Counter-Strike: Global Offensive game. Is that possible for you or would you rather I make a video showing my trials?

circlespainter commented 8 years ago

It would be very difficult for me to try out. How about a very small Java-only test exhibiting the issue, could you write it?

Also, is there no other information available in the exception or elsewhere? Is JNA calling suspendable code or are you instrumenting code in the boot classpath? Asking because you might need Quasar to be in the boot classpath as well (because instrumented classes reference Quasar).

Another troubleshooting attempt that comes to mind is trapping the exception in a debugger and finding out its context.

And instrumentation debugging shows nothing useful in the end?

circlespainter commented 8 years ago

@Jire Have you had a chance to try reproducing it in a Java-only sample?

fab1an commented 7 years ago

I had a problem with this and using FlatMappingReceivePort. I only got this to work by copying the class and replacing the use of com.google.base.Function with kotlin lambdas.

Then instrumentation would scan and add the following line to suspendable-supers:

kotlin.jvm.functions.Function1.invoke(Ljava/lang/Object;)Ljava/lang/Object;

Is it possible that the instrumenter has a problem with the hierarchy of inline-lambda / kotlin-function / guava function / java-function. Maybe they all need to be added to suspendables AND suspendable-supers?