Issue while loading custom class to Bootstrap class loader

[Chaho12]

HI, I have issue with class loading where it cannot find certain libraries.

• It cannot find Jackson library, even if it is included inside Trino
• Opening JAR file has classes, but fails to do so during runtime.

2023-05-22T06:14:15.771Z    ERROR   dispatcher-query-0      io.trino.dispatcher.LocalDispatchQuery  query submitter threw exception
java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
    at com.naver.trino.hook.esSender.<clinit>(esSender.java:24)
    at io.trino.plugin.hive.HivePartitionManager.canPartitionsBeLoaded(HivePartitionManager.java)
    at io.trino.plugin.hive.HivePartitionManager.applyPartitionResult(HivePartitionManager.java:165)

public class HookAgent {
    public static void premain(String agentArguments, Instrumentation instrumentation) {
        File configFile = new File(agentArguments);
        configFile = configFile.getAbsoluteFile();
        Map<String, String> properties = loadEventListenerProperties(configFile);
        String props = Joiner.on(",").withKeyValueSeparator("=").join(properties);

        File temp = null;
        try {
            temp = Files.createTempDirectory("tmp").toFile();
        } catch (IOException e) {
            e.printStackTrace();
        }

        ClassInjector.UsingInstrumentation
                .of(temp, ClassInjector.UsingInstrumentation.Target.BOOTSTRAP, instrumentation)
                .injectRaw(ImmutableMap.<String, byte[]>builder()
                        .put(HivePartitionManagerHookAdvice.class.getName(), ClassFileLocator.ForClassLoader.read(HivePartitionManagerHookAdvice.class))
                        .put(esSender.class.getName(), ClassFileLocator.ForClassLoader.read(esSender.class))
                        .build()
          );
          new AgentBuilder.Default()
                .type(ElementMatchers.named("io.trino.plugin.hive.HivePartitionManager"))
                .transform((builder, typeDescription, classLoader, module, protectionDomain) -> builder
                        .method(ElementMatchers.named("canPartitionsBeLoaded"))
                        .intercept(Advice.withCustomMapping().bind(AgentArguments.class, props).to(HivePartitionManagerHookAdvice.class)))
                .installOn(instrumentation);

Where esSender.java:24 has error at private static final ObjectMapper jackson = new ObjectMapper();.

Any ideas or comments would be helpful on finding out why it saids NoClassDefFoundError

[raphw]

You would need to inject every single linked class. That is not normally recommended. What are you trying to do?

[Chaho12]

You would need to inject every single linked class. That is not normally recommended. What are you trying to do?

Yeah that was the issue. I had to add every single linked class, which is seemly impossible as Jackson is too big.

• I notice only some libraries are read during runtime and some don't. I don't quite understand which libraries can be used or not.

What I wanted to do was, create a separate class file(esSender.java) that sends data to elastic search. Within esSender, we use Jackson to create message string to send to ES.

[raphw]

create an abstract class with a public static field of itself. set that field with a subclass that is not injected. only use jvm types in your abstract class.

[Chaho12]

only use jvm types in your abstract class.

Hmm.. I don't quite understand this part. is there any links that you can share what you mean by jvm types?

[raphw]

types the the java package.

[tirerocket]

create an abstract class with a public static field of itself. set that field with a subclass that is not injected. only use jvm types in your abstract class.

@raphw, do you mind describing this design pattern in more detail? I'm trying to solve the same problem.

[raphw]

You would create a class like this:

public abstract class Dispatcher {
  public static Dispatcher dispatcher;
  public abstract void doDispatch();
}

Once on the boot loader, you can implement this dispatcher from your own class loader and set an instance as the field value. You then callback the method via this field.

[tirerocket]

@raphw thanks for the quick response. How would I access my own class loader from the Agent to create the Dispatcher implementation?

This seems incorrect:

Dispatcher.dispatcher = (Dispatcher) Class.forName("agent.DispatcherImpl", false, customClassLoader) .newInstance();

[tirerocket]

This is what I receive when I do the above:

java.lang.ClassCastException: class agent.DispatcherImpl cannot be cast to class agent.Dispatcher (agent.DispatcherImpl is in unnamed module of loader 'app'; agent.Dispatcher is in unnamed module of loader 'bootstrap')

[tirerocket]

Nevermind - I see what you mean now. Just gotta play with the classloaders. Thanks!

[xuhuanzy]

Nevermind - I see what you mean now. Just gotta play with the classloaders. Thanks!

Could you share it? How do I inject into the dispatcher? I tried to implement my own classloader Dispatcher, but when setting it up, I got an error saying their class loaders are different.