raphw / byte-buddy

Runtime code generation for the Java virtual machine.
https://bytebuddy.net
Apache License 2.0
6.23k stars 804 forks source link

Inconsistent frame length when instrumenting a method #1573

Closed rupinder10 closed 6 days ago

rupinder10 commented 9 months ago

I have a simple setup of instrumenting a method that is working perfectly.

@Retention(RetentionPolicy.RUNTIME)
  public @interface PackageName {
}

@Retention(RetentionPolicy.RUNTIME)
  public @interface ClassName {
}

@Retention(RetentionPolicy.RUNTIME)
  public @interface EntryMethod {
}

public class AgentTransformer implements AgentBuilder.Transformer {
  public Builder<?> transform(Builder<?> builder, TypeDescription typeDescription, ClassLoader classLoader,
                                JavaModule module, ProtectionDomain protectionDomain) {
  { 
     Advice advice = Advice.withCustomMapping().bind(PackageName.class, Constants.packageName)
                       .bind(ClassName.class, Constants.className).bind(EntryMethod.class, Constants.entryMethodName).to(LoopAdvice.class);
     builder = builder.visit(advice.on("invoke"));  
  }
}

public class LoopAdvice {
  @Advice.OnMethodEnter(suppress = Throwable.class)
  public static void onEnter(@Advice.This FlElement element, @Advice.Argument(0) FlState state,
    @Advice.FieldValue(value = "_inarray") Object[] inArray, @PackageName String pkgName,
    @ClassName String className, @EntryMethod String entryMethod) throws Throwable {
      if (inArray == null) {
        ((Method) System.getProperties().get(entryMethod)).invoke((Object) null, element, state);
      }
  }
}

The class and method being instrumented:

public class FlLoop extends FlElement {
    Object[] _inarray;
    public void invoke(FlowState state) {
      //More code here
    }
}

The issue is that I turned on obfuscation using proguard and started getting the error below:

Class com.wm.flow.FlLoop could not be transformed:Inconsistent frame length for public static void com.nbl.agent.advice.reflect.LoopAdvice.onEnter(com.wm.FlElement, com.wm.FlState,java.lang.Object[],java.lang.String,java.lang.String,java.lang.String) throws java.lang.Throwable: 0

The real problem is that I excluded any obfuscation for the AgentTransformer and the LoopAdvice classes and only obfuscated other classes. But the error still persists.

Looking for any suggestions on what would cause this error and what can proguard mess up even when these classes are excluded in the obfuscation.

raphw commented 9 months ago

You can instrument obfuscated code, but your advice classes need to be excluded from obfuscation. The templating mechanism can otherwise no longer use the byte code as a template.

rupinder10 commented 9 months ago

That is the strange part. I have excluded the particular Advice class and also the class that implements the transformer. Here is the setting used:

-keep,allowshrinking public class *AgentTransformer*, *AgentTransformer$* {
    *** *;
    *** *(...);
}

-keep,includedescriptorclasses public class com.nbl.advice.reflect.* {
    *** *;
    *** *(...);
}
raphw commented 9 months ago

If you open the jar, you can use javap to inspect class files. I suspect you'd get different output for advice with obfuscation active.

rupinder10 commented 9 months ago

You are correct. But decompiling both the classes yields logically the same code except that the parameter names for the methods are different. Which I assume should not matter. But I might be wrong based on the logic used in bytebuddy.

I have attached the classes, their decompiled versions and their javap outputs.

cl.zip

raphw commented 9 months ago

The parameter names should not matter, but if the files are altered, the exclusion does not seem to work. I still expect that a full exclusion would do the trick.

rupinder10 commented 9 months ago

The issue is that the advice classes are a part of a jar file and the rest of the classes in the jar need to be obfuscated. So I can't exclude one class in the jar. So the only thing I can do is to control how much gets obfuscated. So I configured proguard to keep the advice classes name, the method and field names and then parameter names too. But the instrumentation still doesnt work.

On Wed, Dec 20, 2023 at 6:02 PM Rafael Winterhalter < @.***> wrote:

The parameter names should not matter, but if the files are altered, the exclusion does not seem to work. I still expect that a full exclusion would do the trick.

— Reply to this email directly, view it on GitHub https://github.com/raphw/byte-buddy/issues/1573#issuecomment-1865254503, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB56UZ6V2HWY632TN2Q6O63YKNU7XAVCNFSM6AAAAABAY5U4H6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRVGI2TINJQGM . You are receiving this because you authored the thread.Message ID: @.***>

dogourd commented 9 months ago

I have encountered this issue before. If I remember correctly, proguard regenerates stack frame information for branch code and exception blocks during the preverify stage. The format of the regenerated information does not match what javac generates, even though it can pass JVM class verification. Currently byte-buddy still cannot properly recognize it.

You cannot configure proguard to ignore this with -keep or similar because it happens during the preverify stage rather than the obfuscate stage. And if you configure proguard to skip verify behavior, you will typically get other strange errors that ultimately cause the obfuscated classes to fail JVM verification.

Ultimately, some known workarounds are:

  1. As discussed previously, keep your Advice code separate from the proguard processed jar, outside of obfuscation, and re-copy it back in after obfuscation is complete. This should can be achieved with a Maven plugin like maven-dependency-plugin.
  2. Keep your Advice code simple, removing any branches or exception blocks - e.g. have just one line of code in the Advice.
  3. Configure ASMVisitorWrapper.writeFlags(ClassWriter.COMPUTE_FRAMES) internally in the Transformer, which will cause ASM to recompute the stack frame information. However this can fail and ultimately prevent successful transformation of the target method if some types are not visible.
rupinder10 commented 9 months ago

Thanks a lot for the detailed response. This makes sense. Because I have several advice files and the only ones that are failing are that have a if statement in the Advice code.

If I remove the if statement I dont get the error. I will try to repackage the jars to avoid this problem.


From: yuwen @.> Sent: Thursday, December 21, 2023 2:33 AM To: raphw/byte-buddy @.> Cc: Rupinder Singh @.>; Author @.> Subject: Re: [raphw/byte-buddy] Inconsistent frame length when instrumenting a method (Issue #1573)

I have encountered this issue before. If I remember correctly, proguard regenerates stack frame information for branch code and exception blocks during the preverify stage. The format of the regenerated information does not match what javac generates, even though it can pass JVM class verification. Currently byte-buddy still cannot properly recognize it.

You cannot configure proguard to ignore this with -keep or similar because it happens during the preverify stage rather than the obfuscate stage. And if you configure proguard to skip verify behavior, you will typically get other strange errors that ultimately cause the obfuscated classes to fail JVM verification.

Ultimately, some known workarounds are:

  1. As discussed previously, keep your Advice code separate from the proguard processed jar, outside of obfuscation, and re-copy it back in after obfuscation is complete. This should can be achieved with a Maven plugin like maven-dependency-plugin.
  2. Keep your Advice code simple, removing any branches or exception blocks - e.g. have just one line of code in the Advice.
  3. Configure ASMVisitorWrapper.writeFlags(ClassWriter.COMPUTE_FRAMES) internally in the Transformer, which will cause ASM to recompute the stack frame information. However this can fail and ultimately prevent successful transformation of the target method if some types are not visible.

— Reply to this email directly, view it on GitHubhttps://github.com/raphw/byte-buddy/issues/1573#issuecomment-1865765266, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB56UZ65JKXF27ZYQR5IOFLYKPQ5RAVCNFSM6AAAAABAY5U4H6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRVG43DKMRWGY. You are receiving this because you authored the thread.Message ID: @.***>