Kotlin anonymous classes provoke errors

[bric3]

package com.github.bric3.ha

import java.util.concurrent.Executors
import java.util.concurrent.TimeUnit

fun main(args: Array<String>) {
    println(Reproducer().o)

    Executors.newSingleThreadScheduledExecutor()
        .scheduleAtFixedRate(
            Runnable {
                println(Reproducer().o)

            },
            2,
            2,
            TimeUnit.SECONDS
        )
}

class Reproducer {
    val o = object {
        override fun toString(): String {
            return "foo-bar"
        }
    }
}

/Users/brice.dutheil/.asdf/installs/java/trava-11.0.11+1/bin/java \
  -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:62176,suspend=y,server=n \
  -XX:HotswapAgent=external \
  -javaagent:hotswap-agent-1.4.2-SNAPSHOT.jar \
  -javaagent:/Users/brice.dutheil/Library/Caches/JetBrains/IntelliJIdea2021.3/captureAgent/debugger-agent.jar=file:/private/var/folders/vr/gc7zwl7x1kvcykpbl0j4zdb00000gq/T/capture.props \
  -Dfile.encoding=UTF-8 \
  -classpath 
/Users/brice.dutheil/ha-kotlin/build/classes/kotlin/main:/Users/brice.dutheil/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk8/1.5.31/ff5d99aecd328872494e8921b72bf6e3af97af3e/kotlin-stdlib-jdk8-1.5.31.jar:/Users/brice.dutheil/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-reflect/1.5.31/1523fcd842a47da0820cea772b19c51056fec8a9/kotlin-reflect-1.5.31.jar:/Users/brice.dutheil/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk7/1.5.31/77e0f2568912e45d26c31fd417a332458508acdf/kotlin-stdlib-jdk7-1.5.31.jar:/Users/brice.dutheil/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.5.31/6628d61d0f5603568e72d2d5915d2c034b4f1c55/kotlin-stdlib-1.5.31.jar:/Users/brice.dutheil/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-common/1.5.31/43331609c7de811fed085e0dfd150874b157c32/kotlin-stdlib-common-1.5.31.jar com.github.bric3.ha.ReproducerKt

This gives the following logs

HOTSWAP AGENT: 17:31:26.524 INFO (org.hotswap.agent.HotswapAgent) - Loading Hotswap agent {1.4.2-SNAPSHOT} - unlimited runtime class redefinition.
HOTSWAP AGENT: 17:31:27.539 INFO (org.hotswap.agent.config.PluginRegistry) - Discovered plugins: [JdkPlugin, ClassInitPlugin, AnonymousClassPatch, WatchResources, Hotswapper, Hibernate, Hibernate3JPA, Hibernate3, Spring, Jersey1, Jersey2, Jetty, Tomcat, ZK, Logback, Log4j2, MyFaces, Mojarra, Omnifaces, ELResolver, WildFlyELResolver, OsgiEquinox, Owb, Proxy, WebObjects, Weld, JBossModules, ResteasyRegistry, Deltaspike, GlassFish, Weblogic, Vaadin, Wicket, CxfJAXRS, FreeMarker, Undertow, MyBatis, IBatis, JacksonPlugin, Idea]
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
foo-bar-qux
HOTSWAP AGENT: 17:32:02.727 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 17:32:02.744 ERROR (org.hotswap.agent.annotation.handler.PluginClassFileTransformer) - InvocationTargetException in transform method on plugin 'class org.hotswap.agent.plugin.jvm.AnonymousClassPatchPlugin' class 'com/github/bric3/ha/Reproducer$o$1'.
java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.hotswap.agent.annotation.handler.PluginClassFileTransformer.transform(PluginClassFileTransformer.java:218)
    at org.hotswap.agent.annotation.handler.PluginClassFileTransformer.transform(PluginClassFileTransformer.java:112)
    at org.hotswap.agent.util.HotswapTransformer.transform(HotswapTransformer.java:246)
    at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:246)
    at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
    at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
Caused by: java.lang.NullPointerException
    at org.hotswap.agent.plugin.jvm.AnonymousClassInfos.lastModified(AnonymousClassInfos.java:275)
    at org.hotswap.agent.plugin.jvm.AnonymousClassInfos.<init>(AnonymousClassInfos.java:109)
    at org.hotswap.agent.plugin.jvm.AnonymousClassPatchPlugin.getStateInfo(AnonymousClassPatchPlugin.java:244)
    at org.hotswap.agent.plugin.jvm.AnonymousClassPatchPlugin.patchAnonymousClass(AnonymousClassPatchPlugin.java:104)
    ... 10 more

HOTSWAP AGENT: 17:32:02.746 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 17:32:02.749 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
foo-bar
foo-bar
foo-bar
foo-bar
foo-bar
foo-bar
Disconnected from the target VM, address: '127.0.0.1:62575', transport: 'socket'

Process finished with exit code 130 (interrupted by signal 2: SIGINT)

The classes are here :

.rw-r--r--   820 brice.dutheil 10 Feb 17:33  Reproducer$o$1.class
.rw-r--r--   862 brice.dutheil 10 Feb 17:33  Reproducer.class
.rw-r--r--  1.9k brice.dutheil 10 Feb 17:33  ReproducerKt.class

[bric3]

I'm not able to debug the agent, but I was able to modify the code and see what was inside the pool.

And on this very reduced reproducer, the class is not in the pool

lastModified called for: com.github.bric3.ha.Reproducer$o, classpool content

• char:org.hotswap.agent.javassist.CtPrimitiveType@1e8ef55c[char]
• long:org.hotswap.agent.javassist.CtPrimitiveType@32ac524b[long]
• float:org.hotswap.agent.javassist.CtPrimitiveType@78f7c35a[float]
• double:org.hotswap.agent.javassist.CtPrimitiveType@6dc64d1a[double]
• short:org.hotswap.agent.javassist.CtPrimitiveType@48b71eb[short]
• byte:org.hotswap.agent.javassist.CtPrimitiveType@5465453b[byte]
• int:org.hotswap.agent.javassist.CtPrimitiveType@124ea1fe[int]
• boolean:org.hotswap.agent.javassist.CtPrimitiveType@4dbf9030[boolean]
• void:org.hotswap.agent.javassist.CtPrimitiveType@12e4b445[void]

[bric3]

Also when developing within IJ for some reasons I had an issue with the AgentLogger, I am not quite why it got wrong.

HOTSWAP AGENT: 16:50:26.254 ERROR (org.hotswap.agent.annotation.handler.PluginClassFileTransformer) - InvocationTargetException in transform method on plugin 'class org.hotswap.agent.plugin.proxy.ProxyPlugin' class 'com/company/intellij/toolpane/SimpleToolWindow$TopList$BackGroundRatioRenderer$jLabel$1'.
java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.hotswap.agent.annotation.handler.PluginClassFileTransformer.transform(PluginClassFileTransformer.java:218)
    at org.hotswap.agent.annotation.handler.PluginClassFileTransformer.transform(PluginClassFileTransformer.java:112)
    at org.hotswap.agent.util.HotswapTransformer.transform(HotswapTransformer.java:246)
    at java.instrument/java.lang.instrument.ClassFileTransformer.transform(ClassFileTransformer.java:246)
    at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
    at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
Caused by: java.lang.NoClassDefFoundError: org/hotswap/agent/logging/AgentLogger
    at org.hotswap.agent.plugin.proxy.hscglib.GeneratorParametersRecorder.<clinit>(GeneratorParametersRecorder.java:38)
Caused by: java.lang.NoClassDefFoundError: org/hotswap/agent/logging/AgentLogger

    at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)
    at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1042)
    at java.base/jdk.internal.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
    at java.base/jdk.internal.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:186)
    at java.base/java.lang.reflect.Field.acquireFieldAccessor(Field.java:1105)
    at java.base/java.lang.reflect.Field.getFieldAccessor(Field.java:1086)
    at java.base/java.lang.reflect.Field.get(Field.java:418)
    at org.hotswap.agent.plugin.proxy.hscglib.GeneratorParametersTransformer.getGeneratorParamsMap(GeneratorParametersTransformer.java:103)
    at org.hotswap.agent.plugin.proxy.hscglib.GeneratorParametersTransformer.getGeneratorParams(GeneratorParametersTransformer.java:130)
    at org.hotswap.agent.plugin.proxy.ProxyPlugin.transformCglibProxy(ProxyPlugin.java:105)
    ... 10 more
Caused by: java.lang.ClassNotFoundException: org.hotswap.agent.logging.AgentLogger PluginClassLoader(plugin=PluginDescriptor(name=PluginName, id=com.company.intellij, descriptorPath=plugin.xml, path=~/company/intellij-plugin/build/idea-sandbox/plugins/plugin-name, version=2.5.6-SNAPSHOT, package=null, isBundled=false), packagePrefix=null, instanceId=120, state=active)
Caused by: java.lang.ClassNotFoundException: org.hotswap.agent.logging.AgentLogger PluginClassLoader(plugin=PluginDescriptor(name=PluginName, id=com.company.intellij, descriptorPath=plugin.xml, path=~/company/intellij-plugin/build/idea-sandbox/plugins/plugin-name, version=2.5.6-SNAPSHOT, package=null, isBundled=false), packagePrefix=null, instanceId=120, state=active)

    at com.intellij.ide.plugins.cl.PluginClassLoader.loadClass(PluginClassLoader.java:235)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
    ... 21 more

[skybber]

From anonymous class plugin's point of view all code must published to class path before redefinition is called. I have noticed a problem with it, when IDE called redefinition before all compilation is synchronized to class path, may be it is the same problem. In my case there was a new class, that did not exist before.

[bric3]

I don't think this is the case here. The patchAnonymousClass methods tries to identify the enclosing class, and it assumes that anonymous classes are suffixed $d (where d is a number) ; this is indeed what javac does, and what Eclipse compiler does (with slight differences in the numbering scheme).

https://github.com/HotswapProjects/HotswapAgent/blob/322b3f7574d85ce67ac379e55e0f1f315ce5ec62/hotswap-agent-core/src/main/java/org/hotswap/agent/plugin/jvm/AnonymousClassPatchPlugin.java#L97-L98

However it seems that kotlinc at this time names the class with additional $identifier it seems (probably to locate the anonymous class).

Hence it seems that indeed the class com.github.bric3.ha.Reproducer$o doesn't exists, but com.github.bric3.ha.Reproducer does.

[bric3]

I have played a bit with kotlin, and discovered they have backticked identifiers. So fun `$$a while - space`(): Any is legal.

So for example the code below will produce these classes :

.rw-r--r--   885 brice.dutheil 11 Feb 00:59  Reproducer$$$a white - space$o$1.class
.rw-r--r--   820 brice.dutheil 11 Feb 00:59  Reproducer$o$1.class
.rw-r--r--  1.2k brice.dutheil 11 Feb 00:59  Reproducer.class
.rw-r--r--  2.0k brice.dutheil 11 Feb 00:59  ReproducerKt.class

import java.util.concurrent.Executors
import java.util.concurrent.TimeUnit

fun main(args: Array<String>) {
    Executors.newSingleThreadScheduledExecutor()
        .scheduleAtFixedRate(
            Runnable {
                print(Reproducer().`$$a white - space`())
                print(" ; ")
                println(Reproducer().o)

            },
            2,
            2,
            TimeUnit.SECONDS
        )
}

class Reproducer {
    val o = object {
        override fun toString(): String {
            return "foo-bar-qux"
        }
    }

    fun `$$a white - space`(): Any {
        val o = object {
            override fun toString(): String {
                return "foo   bar"
            }
        }
        return o
    }
}

So I have made a simple algorithm to find the Kotlin enclosing class:

@@ -101,6 +101,15 @@ public class AnonymousClassPatchPlugin {
         if (classPool.find(className) == null)
             return null;

+        while (classPool.find(mainClass) == null) {
+            // identifiers like this are possible in Kotlin
+            // - val `$$$something` = 1
+            // - val `a white space and a dash -`() { }
+            mainClass = mainClass.substring(0, mainClass.lastIndexOf('$'));
+            System.err.println("Infer mainclass, trying " + mainClass);
+        }
+
         AnonymousClassInfos info = getStateInfo(classLoader, classPool, mainClass);

         String compatibleName = info.getCompatibleTransition(javaClass);

With this change the NPE is gone, however now the replacement does not behave as expected.

eg, if I change

             Runnable {
                 print(Reproducer().`$$a white - space`())
-                print(" ; ")
+                print(" # ")
                 println(Reproducer().o)
             },

The program don't print the correct toString

Connected to the target VM, address: '127.0.0.1:58057', transport: 'socket'
HOTSWAP AGENT: 01:23:02.607 INFO (org.hotswap.agent.HotswapAgent) - Loading Hotswap agent {1.4.2-SNAPSHOT} - unlimited runtime class redefinition.
HOTSWAP AGENT: 01:23:02.892 INFO (org.hotswap.agent.config.PluginRegistry) - Discovered plugins: [JdkPlugin, ClassInitPlugin, AnonymousClassPatch, WatchResources, Hotswapper, Hibernate, Hibernate3JPA, Hibernate3, Spring, Jersey1, Jersey2, Jetty, Tomcat, ZK, Logback, Log4j2, MyFaces, Mojarra, Omnifaces, ELResolver, WildFlyELResolver, OsgiEquinox, Owb, Proxy, WebObjects, Weld, JBossModules, ResteasyRegistry, Deltaspike, GlassFish, Weblogic, Vaadin, Wicket, CxfJAXRS, FreeMarker, Undertow, MyBatis, IBatis, JacksonPlugin, Idea]
foo   bar  qux ; foo-bar-qux
foo   bar  qux ; foo-bar-qux
foo   bar  qux ; foo-bar-qux
foo   bar  qux ; foo-bar-qux
foo   bar  qux ; foo-bar-qux
foo   bar  qux ; foo-bar-qux
foo   bar  qux ; foo-bar-qux
HOTSWAP AGENT: 01:23:18.138 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 01:23:18.146 INFO (org.hotswap.agent.plugin.jvm.AnonymousClassPatchPlugin) - Creating new infos for className com.github.bric3.ha.Reproducer
Infer mainclass, trying com.github.bric3.ha.Reproducer
HOTSWAP AGENT: 01:23:18.155 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 01:23:18.157 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 01:23:18.159 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
com.github.bric3.ha.Reproducer$$$a white - space$o$1@798d04f6 # com.github.bric3.ha.Reproducer$o$1@3dc8d3ae
com.github.bric3.ha.Reproducer$$$a white - space$o$1@3359eaa4 # com.github.bric3.ha.Reproducer$o$1@2c305661
com.github.bric3.ha.Reproducer$$$a white - space$o$1@3d8b4da5 # com.github.bric3.ha.Reproducer$o$1@46a5bc19
com.github.bric3.ha.Reproducer$$$a white - space$o$1@3a5572a3 # com.github.bric3.ha.Reproducer$o$1@1478b8d6
Disconnected from the target VM, address: '127.0.0.1:58057', transport: 'socket'

[bric3]

I wondered if this last bit was something particular to the toString method, but even something different brings unexpected results. In this example I am using a JLabel.

import java.util.concurrent.Executors
import java.util.concurrent.TimeUnit
import javax.swing.JLabel

fun main(args: Array<String>) {
    Executors.newSingleThreadScheduledExecutor()
        .scheduleAtFixedRate(
            Runnable {
                print(Reproducer().`$$a white - space`().text)
                print(" - ")
                println(Reproducer().o.text)
            },
            2,
            2,
            TimeUnit.SECONDS
        )
}

class Reproducer {
    val o = object : JLabel() {
        override fun getText(): String {
            return "Hello from o"
        }
    }

    fun `$$a white - space`(): JLabel {
        val o = object : JLabel() {
            override fun getText(): String {
                return "Hello from white space"
            }
        }
        return o
    }
}

Again just changing this line

             Runnable {
                 print(Reproducer().`$$a white - space`())
-                print(" ; ")
+                print(" # ")
                 println(Reproducer().o)
             },

Produces this

OTSWAP AGENT: 01:31:14.603 INFO (org.hotswap.agent.HotswapAgent) - Loading Hotswap agent {1.4.2-SNAPSHOT} - unlimited runtime class redefinition.
HOTSWAP AGENT: 01:31:14.931 INFO (org.hotswap.agent.config.PluginRegistry) - Discovered plugins: [JdkPlugin, ClassInitPlugin, AnonymousClassPatch, WatchResources, Hotswapper, Hibernate, Hibernate3JPA, Hibernate3, Spring, Jersey1, Jersey2, Jetty, Tomcat, ZK, Logback, Log4j2, MyFaces, Mojarra, Omnifaces, ELResolver, WildFlyELResolver, OsgiEquinox, Owb, Proxy, WebObjects, Weld, JBossModules, ResteasyRegistry, Deltaspike, GlassFish, Weblogic, Vaadin, Wicket, CxfJAXRS, FreeMarker, Undertow, MyBatis, IBatis, JacksonPlugin, Idea]
Hello from white space # Hello from o
Hello from white space # Hello from o
HOTSWAP AGENT: 01:31:40.544 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 01:31:40.554 INFO (org.hotswap.agent.plugin.jvm.AnonymousClassPatchPlugin) - Creating new infos for className com.github.bric3.ha.Reproducer
HOTSWAP AGENT: 01:31:40.567 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 01:31:40.573 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
HOTSWAP AGENT: 01:31:40.576 INFO (org.hotswap.agent.plugin.jdk.JdkPlugin) - Removing class from declaredMethodCache.
 - 
 - 
 - 

It's like the method overrides of the anonymous class are gone.

[scosenza]

Using a recent version HotswapPlugin (compiled from head a few weeks ago), I need to disable the AnonymousClassPatch plugin for any projects using Kotlin. Is this the only option at this point, or do others have any fixes or workarounds? Thanks for any pointers!

[skybber]

As far as the issue causing the NullPointerException in the org.hotswap.agent.plugin.jvm.AnonymousClassInfos.lastModified method has been resolved as of today.

[scosenza]

Amazing news! Thanks so much for all the recent fixes!!!

[skybber]

I recommend to use also the last jbr17/21 versions since there is fixed following problem:

https://youtrack.jetbrains.com/issue/JBR-6647/XXAllowEnhancedClassRedefinition-does-not-work-with-Kotlin