[BUG] Tools can not be a proxy class

[G0dC0der]

Describe the bug Whenever a tool is a proxy class, the LLM is suddenly not aware of them, and they are never invoked.

Log and Stack trace n/a

To Reproduce Basically, whenever you register your tool to an AI service, just wrap the tool around the response of this method:

import org.springframework.aop.framework.ProxyFactory;

class ProxyUtils {

    public static <T> T createProxy(T target) {
        var proxyFactory = new ProxyFactory(target);
        proxyFactory.setProxyTargetClass(true); // Use CGLIB proxying
        return (T) proxyFactory.getProxy();
    }
}

Expected behavior I expect the LLM to be aware of the tools that reside in a proxy class and invoke them accordingly.

Please complete the following information:

• LangChain4j version: 0.32.0
• LLM(s) used: OpenAI
• Java version: 21
• Spring Boot version 3.3.1

Additional context Having tools as proxy classes can potentially save you from writing a lot of boilerplate code and help you streamline certain logic and behaviour.

[langchain4j]

Hi @G0dC0der, do you already have a solution for this issue?

[G0dC0der]

I could try to submit a pull request if you could point me to where this is handled.

[langchain4j]

Sounds great, the starting point is AiServices.tools(...) methods

[G0dC0der]

I don't think supplying a pull request with a fix is the right approach, because that would drag spring into this project(at least Spring AOP lib), which is probably not a good idea. Although I will try to create a neat util function that creates a ToolSpecification from a proxy class.

[langchain4j]

I am not sure what exactly you are trying to achieve, but if it is Spring Boot related, it can be handled in the SB starter.

Although in SB starter we automatically detect all tools (all beans with methods annotated with @Tool) and configure them for AI Service.

Could you please explain your use case a bit more? How do you use proxies? Thanks!

[G0dC0der]

Ah, SB starter is probably the right place to put it. However, Spring Beans are not proxy classes, and the code you are referring to wont work either, because proxy classes(at least those created by CGLIB, which is the most common way to create them), do not inherit annotations. So basically, whenever you create the tool specification, you have to:

1) Check if we're dealing with a proxy class, using some util function in Spring AOP 2) If true, unwrap the proxy class and create a specification from the underlying class(the tool executor should however call the proxy class).

It's not hard nor complex to fix, it will however, require a dependency on Spring AOP, which depends on CGLIB and AspectJ. Not sure if this is a problem.

Why Proxy Class? Basically, a proxy class is a dynamic method interceptor, and this is how I use it:

Instead of doing the following to every single tool:

@Tool 
public void myTool(String param1, String param2)
try {
    // tool code
} catch (CmsException e) {
    //Handle cms exception
} catch (Exception e) {
    //Handle other exception
}

I just create a method interceptor, that runs on ever method annotated with @Tool. This saves me from immense boilerplate(i e the try catch).

[langchain4j]

@G0dC0der ok, what about introducing exception handler in tools as a core feature instead? I guess it will be useful for many users.

AiServices.builder(...)
  .tools(...)
  .toolExceptionHandler(...)
  .build();

[G0dC0der]

How about

public interface ToolInterceptor {
    Object intercept(Method method, Object instance, Object[] args);
}

AiServices.builder(...)
  .tools(...)
  .toolInterceptor(new ToolExceptionHandler())
  .build();

And can be used like this(in my case):

public class ToolExceptionHandler implements ToolInterceptor {
    public Object intercept(Method method, Object instance, Object[] args) {
        try {
            return method.invoke(instance, args); //Calls the tool with the args provided by the tool executor
        } catch(CmsException e) { 
            //Handle cms exception
        }  catch(Exception e) { 
            //Handle the rest
        }
    }
}

The first two param of ToolInterceptor is already present in dev.langchain4j.service.tool.DefaultToolExecutor. There is also a method for getting the args(returning Object[]). So this is the perfect class to store and run them:

https://github.com/langchain4j/langchain4j/blob/main/langchain4j/src/main/java/dev/langchain4j/service/tool/DefaultToolExecutor.java#L85

[langchain4j]

How do you plan to handle exceptions?

[G0dC0der]

Well I learned that exception thrown in tools are propagated to the LLM. I see no reason to change that(meaning exceptions thrown in a tool interceptor are also sent to the LLM. You could also return a static value too).

public class ToolExceptionHandler implements ToolInterceptor {
    public Object intercept(Method method, Object instance, Object[] args) {
        try {
            return method.invoke(instance, args); //Calls the tool with the args provided by the tool executor
        } catch(CmsException e) { 
            return "Some default object"; 
        }  catch(Exception e) { 
            throw new AbortException("A tool failed, please abort any pending operation!"); //A message to the LLM
        }
    }
}

These exceptions(AbortException) are propagated to the LLM, just as the current behaviour of a tool.

Did this answer your question?

[langchain4j]

Yes, thank you.

I would just wrap input and output into objects to be able to extend them with more fields in the future:

ToolInterceptionResult intercept(ToolInterceptionRequest request);

[langchain4j]

@G0dC0der tools wrapped into AOP proxies are now supported: https://github.com/langchain4j/langchain4j-spring/pull/80 This will be released in 0.37.0 and is already available in 0.37.0-SNAPSHOT