Agent creation with common dependencies : LinkageError

[allemas]

Hello,

I'm trying to instrument the vert.x context to manually add the span trace as OpenTelemetry does.

I'm facing a classpath error that I can't fix correctly.

Caused by: java.lang.LinkageError: loader constraint violation: when resolving method 'io.vertx.core.Handler com.byteprofile.bytecodr.agent.instrumentation.vertx.HandlerWrapper.wrap(io.vertx.core.Handler)' the class loader io.quarkus.bootstrap.runner.RunnerClassLoader @386f0da3 of the current class, io/vertx/ext/web/impl/RouteImpl, and the class loader 'app' for the method's defining class, com/byteprofile/bytecodr/agent/instrumentation/vertx/HandlerWrapper, have different Class objects for the type io/vertx/core/Handler used in the signature (io.vertx.ext.web.impl.RouteImpl is in unnamed module of loader io.quarkus.bootstrap.runner.RunnerClassLoader @386f0da3, parent loader 'app'; com.byteprofile.bytecodr.agent.instrumentation.vertx.HandlerWrapper is in unnamed module of loader 'app')

In the OTEL repository the HandlerWrapper implement Handler<T>. who's a vertx dependency.

I'm quite sure I should isolate classpath for agent and quarkus, is this really the right thing to do? Do you have some examples or pointers ?

Here my ByteBuddy agent, maybe I miss something ?

import io.vertx.core.Handler;

new AgentBuilder
                .Default()
                .ignore(ElementMatchers.nameStartsWith("net.bytebuddy."))
                .with(RETRANSFORMATION) 
                .disableClassFormatChanges()
                .with(AgentBuilder.InstallationListener.StreamWriting.toSystemError())  
                .type(is(RouteImpl.class))
                .transform(new AgentBuilder.Transformer.ForAdvice()
                        .advice(ElementMatchers.named("handler")
                                        .and(takesArgument(0, named("io.vertx.core.Handler"))),
                                HandlerVisitorCallSite.class.getName()
                        ))
                ;

public class HandlerVisitorCallSite {
    @Advice.OnMethodEnter()
    public static void onEnter(
            @Advice.Argument(value = 0, readOnly = false, typing = Assigner.Typing.DYNAMIC) Handler<?> handler) {
        System.out.print("ENTER the method: " + '\n');
        Handler<?> handler = HandlerWrapper.wrap(handler);
    }
}

import io.vertx.core.Handler;

public class HandlerWrapper<T> implements Handler<T> {
    private final Handler<T> delegate;

    private HandlerWrapper(Handler<T> delegate) {
        this.delegate = delegate;
    }

    public static <T> Handler<T> wrap(Handler<T> handler) {
        handler = new HandlerWrapper<>(handler);
        return handler;
    }

    @Override
    public void handle(T t) {
        delegate.handle(t);
    }
}

Thank you very much for your time

[raphw]

Instead of is(RouteImpl.class), use named("io.vertx.RouteImpl"), assuming this is the right package. Avoid loading user classes from your agent as far as possible.

[allemas]

Nop it doesn't work, the error probably comes from the fact that I'm using Handler<T> and that it's also used in quarkus

[raphw]

Yes, indeed on a second look the problem is likely that you contain vertx in your agent and that the handler class is loaded twice. What you need to do: exclude the vertx dependencies from your agent, and avoid calling the vertx classes from it. You should only depend on it for compiling the advice and defining the helper classes.

Resolving the advice is already handled by Byte Buddy, but in your transform method, you will also need to inject the helper class. You can do so by using a ClassInjector, but I just tried to make this more convenient by adding auxiliary methods to the ForAdvice class. You can build Byte Buddy from source if you wanted to use this, or wait for the next release.

[allemas]

I think what I'm looking for is something resembling as a dynamic invocation mechanism as described in the AssignReturned chapter: https://www.elastic.co/fr/blog/embracing-invokedynamic-to-tame-class-loaders-in-java-agents .

I already have the shared class loader part with :

new AgentBuilder.Transformer.ForAdvice().include(classLoader)

but now when I run the agent in the application, I get a java.lang.NoClassDefFoundError problem when the application tries to create a class that implements Handler

I think the answer is around net.bytebuddy.dynamic.loading.MultipleParentClassLoader and Advice.withCustomBinding.

Just to clarify, in the latest version of bytebuddy, is the Advice.withCustomBinding method replaced by Advice.withCustomMapping()?

                     new AgentBuilder.Transformer.ForAdvice()
                        .include(classLoader) 
                        .advice(ElementMatchers.named("handler")), Advice.withCustomMapping().....)

I'm afraid I'm trying to use it the wrong way.

[raphw]

If you run with the latest version of Byte Buddy, you can add: .auxiliary("com.acme.HandlerWrapper") what should solve your issue.

[allemas]

I maybe find a solution:

• The instrumentation is in a jar, and is shaded.
• In the agent jar I have no dependencies, only its own code (no common dependencies)
• I create a custom ClassLoader in the agent (from premain method), which is responsible for loading the jar (by it's absolute path for now)
• Use this classpath for Advice method as here

  new AgentBuilder.Transformer.ForAdvice()
                      .include(classLoader) 

• I don't append instrumentation instance with this class classloader (no conflict)

Now I'm facing an other issue,

Caused by: java.lang.ClassNotFoundException: com.byteprofile.HandlerWrapper

It's quite logical, given that I'm instrumenting with the agent jar which doesn't have the dependencies, so on retransformation the classes aren't inlined.

Is there any way of defining a class loading strategy? I've already tried to create a JAR without the dependencies for the ‘vert.x instrumentation’ part but I end up with a java.lang.ClassNotFoundException on the original io.vertx.core.Handler :(

I don't think that's the right method, I suppose it's better to create a strategy that injects the ClassLoader used in the Advice method into as sub ClassLoader, and avoid conflict

I believe there is .with(ClassLoadingStrategy....) but I'm not sure of my choices, maybe .with(Default.WRAPPER.load(CL,)) ? or RawMatcher ?

Here an example of my code : https://github.com/allemas/bytecodr-for-vertx/blob/main/vert-x-instrumentation/src/main/java/com/byteprofile/InstrumentFactory.java

[raphw]

Did you try .auxiliary("com.byteprofile.HandlerWrapper")? What exception does that yield?

[allemas]

I've just realized that I wasn't in the latest version of Bytebuddy my bad.

I've just added the method, it works better but I still have an exception.

                .transform(new AgentBuilder.Transformer.ForAdvice()
                        .include(cusctomClassLoader)
                        .advice(ElementMatchers.named("handler")
                                .and(takesArgument(0, named("io.vertx.core.Handler")
                                )), "com.byteprofile.instrumentation.HandlerVisitorCallSite")
                        .auxiliary(List.of("io.vertx.core.Handler", "com.byteprofile.instrumentation.HandlerVisitorCallSite"))
                )

Instrumentation is fine :

[Byte Buddy] DISCOVERY io.vertx.ext.web.impl.RouteImpl [io.quarkus.bootstrap.runner.RunnerClassLoader@1, unnamed module @55ca8de8, Thread[#1,main,5,main], loaded=false]
[Byte Buddy] TRANSFORM io.vertx.ext.web.impl.RouteImpl [io.quarkus.bootstrap.runner.RunnerClassLoader@1, unnamed module @55ca8de8, Thread[#1,main,5,main], loaded=false]

But I get an exception when the application executes the code:

2024-11-10 00:41:38,374 ERROR [io.qua.run.Application] (main) Failed to start application: java.lang.RuntimeException: Failed to start quarkus
        at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
        at io.quarkus.runtime.Application.start(Application.java:101)
        at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:119)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
        [....]
Caused by: java.lang.NoClassDefFoundError: io/vertx/core/Handler

However, in the class I display the classloader and it's really quarkus's :(

public class HandlerVisitorCallSite {
    @Advice.OnMethodEnter
    public static void onEnter(
            @Advice.Argument(value = 0, readOnly = false) Handler<RoutingContext> handler
    ) {
        System.out.println("ENTER the method: ");
        try {
            System.out.println("-->" + handler.getClass().getClassLoader());
            handler = new HandlerWrapper(handler);
        } catch (Exception e) {
            System.out.println(e.getMessage());
            throw new RuntimeException(e);
        }
    }
 }

And I got

ENTER the method: 
-->io.quarkus.bootstrap.runner.RunnerClassLoader@1

Thank you so much for taking the time to help me. I truly appreciate your input and effort in supporting my experiments.

[raphw]

Of course. Quarkus is using Graal native image and class loaders behave a bit different there. Does this work without Quarkus? The injection targets the contextual class loader.

[allemas]

Sorry for the delay! Yes, it seems to work as I expect.

I guess with Quarkus we have a classloader issue, that when we execute the agent, the modified class is not dynamically loaded yet. And then, once io.quarkus.bootstrap.runner.RunnerClassLoader has finished its work, we have the same class twice, the first: the class injected by the agent and the second: the one loaded by the RunnerClassLoader.