How to cast the actual parametric return type of a method belonging to a generic type?

[stechio]

Info	Value
Artifact	net.bytebuddy:byte-buddy:1.15.10 (LATEST)
JRE	11

Context

I am extending a third-party application which dynamically loads extensions via URLClassLoader.

My extension encompasses two artifacts (say, ext1.jar and ext2.jar), each implementing a distinct kind of functionality (say, MyExtension1 and MyExtension2 implementing Extension1 and Extension2 application domain interfaces, along with respective ancillary types); ext2.jar depends on ext1.jar, so the public types of ext1.jar (i.e., MyExtension1 and its ancillary types) are visible to ext2.jar.

As the third-party application manages extensions by functionality, those 2 artifacts are loaded separately, along with their respective dependencies, so the resulting class loader hierarchy is:

• Application domain types:

  jdk.internal.loader.ClassLoaders$PlatformClassLoader@1d29cf23 jdk.internal.loader.ClassLoaders$AppClassLoader@1dbd16a6

• MyExtension1 (implements Extension1) types:

  jdk.internal.loader.ClassLoaders$PlatformClassLoader@1d29cf23 jdk.internal.loader.ClassLoaders$AppClassLoader@1dbd16a6 java.net.URLClassLoader@36f6e879 // urlClassLoader1

• MyExtension2 (implements Extension2, depends on MyExtension1) types:

  jdk.internal.loader.ClassLoaders$PlatformClassLoader@1d29cf23 jdk.internal.loader.ClassLoaders$AppClassLoader@1dbd16a6 java.net.URLClassLoader@126253fd // urlClassLoader2

As a consequence, when the logic of MyExtension2 retrieves from the application domain an instance of Extension1 (whose actual type is MyExtension1@urlClassLoader1) and tries to cast it as MyExtension1@urlClassLoader2, a typical ClassCastException occurs:

MyExtension1 myExt1 = App.getExtension1("MyExtension1"); /* <-- ClassCastException:
                                      `App` returns MyExtension1@urlClassLoader1,
                                      while `MyExtension1` variable type is resolved as
                                      MyExtension1@urlClassLoader2. */

Since the loading logic of the application is beyond my control, I cannot fix such type split but work around it: the ugly serialization/deserialization trick is out of the question (I need dynamic interaction, not state copy!), so reflection seems the only viable option. However, since explicit reflection would be a royal pain, I decided to employ proxying via ByteBuddy.

Issue

So far, my solution leveraging ByteBuddy has proved to work quite well at pre-alpha stage (kudos to @raphw!), except for one annoying issue I haven't been able to solve yet: casting of parametric return types for generic interfaces. For example:

/*
  NOTE: `xcast(..)` is the proxying method (see its code below).
*/
var ext1s = xcast(App.getExtension1s(), null);
var itr1 = ext1s.entrySet().iterator();
Map.Entry entry = itr1.next(); /* <-- ClassCastException:
                                class net.bytebuddy.renamed.java.lang.Object$ByteBuddy$gBLbFcx4
                                cannot be cast to class java.util.Map$Entry */
System.out.println("ext1 name: " + entry.getValue().getName());

The ClassCastException here above is due to the circumstance that the proxied T next() uses the result of baseMethod.getReturnType() (see code below) as proxied type (which is obviously erased to Object), instead of the actual type parameter of Iterator<Map.Entry>.

Here it is my proxying logic:

public static <T> T xcast(Object obj, Class<?> objType) {
  final Class<?> sourceType = objType != null ? objType : obj.getClass();
  final Class<?> targetType = Class.forName(sourceType.getName());
  if (targetType == sourceType
      && (isPrimitiveWrapper(targetType) || targetType == String.class))
    return (T) obj;

  Class<? extends T> proxyType = new ByteBuddy()
      .subclass(targetType, ConstructorStrategy.Default.NO_CONSTRUCTORS)
      .defineField("proxyBase", Object.class, Modifier.PUBLIC + Modifier.FINAL)
      .defineConstructor(Visibility.PUBLIC)
      .withParameters(Object.class)
      .intercept(MethodCall.invoke(targetType.isInterface()
            ? Object.class.getDeclaredConstructor()
            : targetType.getDeclaredConstructor()).onSuper()
          .andThen(FieldAccessor.ofField("proxyBase").setsArgumentAt(0)))
      .method(ElementMatchers.any())
      .intercept(InvocationHandlerAdapter.of(new InvocationHandler() {
        @Override
        public Object invoke(Object proxy, Method method, Object[] args)
            throws Throwable {
          // Retrieve the source object associated to this proxy instance!
          var base = proxy.getClass().getDeclaredField("proxyBase").get(proxy);

          // Get the source method corresponding to the invoked proxy method!
          /*
           * NOTE: This is necessary as (proxy) `method` is binary-incompatible with
           * (source) `base`.
           */
          var baseMethod = sourceType.getMethod(method.getName(), method.getParameterTypes());

          // Delegate the invocation to the source object!
          /*
           * NOTE: Return value is cross-cast in turn, to ensure any binary-incompatible
           * type is encapsulated into its own proxy.
           */
          return xcast(baseMethod.invoke(base, args), baseMethod.getReturnType());
        }
      }))
      .make()
      .load(Temp.class.getClassLoader())
      .getLoaded();

  return proxyType.getConstructor(Object.class).newInstance(obj);
}

(NOTE: proxy caching logic has been omitted for the sake of clarity)

I read something about withAssigner(Assigner.DEFAULT, Assigner.Typing.DYNAMIC), for example on stackoverflow, but I'm unsure whether it is appropriate for my use case; furthermore, apparently it is not applicable to invocation handlers... Out of desperation, I gave a try to withAssigner(Assigner.GENERICS_AWARE), but failed ("java.lang.UnsupportedOperationException: Assignability checks for type variables declared by methods are not currently supported").

Is there a convenient way to cast the parametric return type of a method belonging to a generic type, so the actual type of that parameter is returned instead of its erasure? thanks!

[raphw]

You are falling victim to type erasure, where baseMethod.getReturnType() returns a type without considering its generic form. Why don't you use the type of the returned value of baseMethod.invoke(base, args) as type argument of xcast?

[stechio]

@raphw:

You are falling victim to type erasure, where baseMethod.getReturnType() returns a type without considering its generic form. Why don't you use the type of the returned value of baseMethod.invoke(base, args) as type argument of xcast?

Because I have no guarantees that the type of the value returned by baseMethod.invoke(base, args) has a no-argument constructor (which is required for proxy subclassing, otherwise "java.lang.RuntimeException: java.lang.NoSuchMethodException: XXXXXXX.<init>()"), so anytime possible I prefer baseMethod.getReturnType(), which is typically (at least in my domain) an interface (which is ideal for proxy "subclassing" (more appropriately, implementation), as it's intrinsically costructor-free and allows me to call the no-argument constructor of Object).

I temporarily jury-rigged the type erasure problem picking the first interface available, but it's a horrible (although effective) hack:

public static <T> T xcast(Object obj, Class<?> objType) {
  final Class<?> sourceType;
  if (objType == Object.class) {
    // Look for the first interface (if any) to implement as proxy!
    objType = obj.getClass();
    var candidateSourceType = objType;
    while (objType != Object.class) {
      var interfaceTypes = objType.getInterfaces();
      if (interfaceTypes.length > 0) {
        candidateSourceType = interfaceTypes[0];
        break;
      }

      objType = objType.getSuperclass();
    }
    sourceType = candidateSourceType;
  } else {
    sourceType = objType != null ? objType : obj.getClass();
  }
  . . .
}

I hoped a more robust solution could possibly be exploited through bytecode manipulation, alas... :cry:

[raphw]

You can wrap classes in a TypeDescription and navigate their generic hierarchy using Byte Buddy which resolves type variables and the like transparently. Apart from that, Byte Buddy is bound by the JVM as anything else, so your approach might be the best way of solving it, depending on your domain.

[stechio]

@raphw:

You can wrap classes in a TypeDescription and navigate their generic hierarchy using Byte Buddy which resolves type variables and the like transparently.

Is there any tutorial/demo which provides a context to grasp the proper usage of TypeDescription (I couldn't find any documentation other than javadoc)? thanks!

[raphw]

Simply load a class using TypeDescription.ForLoadedType.of(...). Then navigate the hierarchy as you would with the reflection API where generic types are resolved transparently. To resolve the return type of methods, use MethodGraph.Compiler.

[stechio]

Thank you very much for your hints!

Your library is powerful and its API is slick, but its breadth is a bit intimidating to get started with — maybe something like a community-driven repository of practical code snippets, like a cookbook, would help lowering the learning curve :sweat_smile: Thanks again