search

raphw / byte-buddy

Runtime code generation for the Java virtual machine.
https://bytebuddy.net
Apache License 2.0
6.29k stars 807 forks source link

ByteBuddy 1.3.0 occur exception，when our team uses in javaagent #162

Closed wu-sheng closed 8 years ago

wu-sheng commented 8 years ago

First of all, thanks for your project. We build a very popular java APM(Application Performance Management) project named SkyWalking(https://github.com/wu-sheng/sky-walking) based on ByteBuddy.

Today, we meet an issues as follow:

    public static void premain(String agentArgs, Instrumentation instrumentation) throws PluginException {
        initConfig();
        if (AuthDesc.isAuth()) {
            List<IPlugin> plugins = new PluginBootstrap().loadPlugins();
            logger.info("Loaded " + plugins.size() + " plugin");
            final PluginDefineCategory pluginDefineCategory = PluginDefineCategory.category(plugins);

            startBootPluginDefines(pluginDefineCategory.getBootPluginsDefines());

            new AgentBuilder.Default().type(exclusivePackageClass()).transform(new AgentBuilder.Transformer() {
                public DynamicType.Builder<?> transform(DynamicType.Builder<?> builder, TypeDescription typeDescription, ClassLoader classLoader) {
                    AbstractClassEnhancePluginDefine pluginDefine = pluginDefineCategory.getClassEnhancePluginDefines().get(typeDescription.getTypeName());
                    if (pluginDefine == null) {
                        return builder;
                    }

                    try {
                        return pluginDefine.define(builder);
                    } catch (Throwable e) {
                        e.printStackTrace();
                        logger.error("Failed to enhance plugin " + pluginDefine.getClass().getName(), e);
                        return builder;
                    }
                }
            }).installOn(instrumentation);

        }
    }
/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/bin/java -javaagent:/Users/wusheng/Documents/code/github/sky-walking/skywalking-collector/skywalking-agent/target/skywalking-agent-1.0-Final.jar -Didea.launcher.port=7532 "-Didea.launcher.bin.path=/Applications/IntelliJ IDEA.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/lib/tools.jar:/Users/wusheng/Documents/code/github/sky-walking/samples/skywalking-example/example-dubbo/dubbo-impl/target/classes:/Users/wusheng/Documents/code/github/sky-walking/samples/skywalking-example/example-dubbo/dubbo-interfaces/target/classes:/Users/wusheng/Documents/work/00_devtools/m2/com/alibaba/dubbox/2.8.4/dubbox-2.8.4.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/javassist/javassist/3.18.1-GA/javassist-3.18.1-GA.jar:/Users/wusheng/Documents/work/00_devtools/m2/com/101tec/zkclient/0.7/zkclient-0.7.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar:/Users/wusheng/Documents/work/00_devtools/m2/log4j/log4j/1.2.15/log4j-1.2.15.jar:/Users/wusheng/Documents/work/00_devtools/m2/javax/mail/mail/1.4/mail-1.4.jar:/Users/wusheng/Documents/work/00_devtools/m2/javax/activation/activation/1.1/activation-1.1.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.jar:/Users/wusheng/Documents/work/00_devtools/m2/jline/jline/0.9.94/jline-0.9.94.jar:/Users/wusheng/Documents/work/00_devtools/m2/io/netty/netty/3.7.0.Final/netty-3.7.0.Final.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/springframework/spring-beans/4.3.1.RELEASE/spring-beans-4.3.1.RELEASE.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/springframework/spring-core/4.3.1.RELEASE/spring-core-4.3.1.RELEASE.jar:/Users/wusheng/Documents/work/00_devtools/m2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/springframework/spring-tx/4.3.1.RELEASE/spring-tx-4.3.1.RELEASE.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/apache/logging/log4j/log4j-core/2.6.2/log4j-core-2.6.2.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/apache/logging/log4j/log4j-api/2.6.2/log4j-api-2.6.2.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/springframework/spring-jdbc/4.3.1.RELEASE/spring-jdbc-4.3.1.RELEASE.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/apache/commons/commons-dbcp2/2.1.1/commons-dbcp2-2.1.1.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/apache/commons/commons-pool2/2.4.2/commons-pool2-2.4.2.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/mybatis/mybatis-spring/1.3.0/mybatis-spring-1.3.0.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/springframework/spring-context/4.3.1.RELEASE/spring-context-4.3.1.RELEASE.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/springframework/spring-aop/4.3.1.RELEASE/spring-aop-4.3.1.RELEASE.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/springframework/spring-expression/4.3.1.RELEASE/spring-expression-4.3.1.RELEASE.jar:/Users/wusheng/Documents/work/00_devtools/m2/com/h2database/h2/1.4.192/h2-1.4.192.jar:/Users/wusheng/Documents/work/00_devtools/m2/org/mybatis/mybatis/3.4.1/mybatis-3.4.1.jar:/Users/wusheng/Documents/code/github/sky-walking/skywalking-collector/skywalking-log/log4j-1.x-plugin/target/classes:/Users/wusheng/Documents/code/github/sky-walking/skywalking-collector/skywalking-api/target/classes:/Users/wusheng/Documents/code/github/sky-walking/skywalking-collector/skywalking-protocol/target/classes:/Users/wusheng/Documents/work/00_devtools/m2/com/google/protobuf/protobuf-java/2.6.1/protobuf-java-2.6.1.jar:/Users/wusheng/Documents/work/00_devtools/m2/net/bytebuddy/byte-buddy/1.3.0/byte-buddy-1.3.0.jar:/Users/wusheng/Documents/work/00_devtools/m2/io/netty/netty-all/4.0.33.Final/netty-all-4.0.33.Final.jar:/Users/wusheng/Documents/work/00_devtools/m2/com/google/code/gson/gson/2.7/gson-2.7.jar:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar" com.intellij.rt.execution.application.AppMain com.ai.cloud.skywalking.sample.util.DubboStart
objc[73211]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
Exception in thread "Monitor Ctrl-Break" 
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "Monitor Ctrl-Break"
Exception in thread "main" 
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "main"
Java HotSpot(TM) 64-Bit Server VM warning: Exception java.lang.NoClassDefFoundError occurred dispatching signal SIGINT to handler- the VM may need to be forcibly terminated
wusheng:tmp wusheng$ java -version
java version "1.7.0_67"
Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

Now, we tried many ways. The only way to avoid this issue, is add a matcher and exclude package 'sun.*', like this:

new AgentBuilder.Default().type(**not(ElementMatchers.nameStartsWith("sun"))**).transform(new AgentBuilder.Transformer()

We hope this issue could be fixed soon enough. Thank you. @wu-sheng @ascrutae

wu-sheng commented 8 years ago

I try to use lastest version 1.4.6 to fixed this issue. I get another exception, when the application uses log4j.(remove agent, everything is fine. )

objc[75496]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/bin/java and /Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/jre/lib/libinstrument.dylib. One of the two will be used. Which one is undefined.
Exception in thread "main" java.lang.NoClassDefFoundError: sun/reflect/GeneratedMethodAccessor4
    at sun.reflect.GeneratedMethodAccessor4.<clinit>(Unknown Source)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
    at java.lang.Class.newInstance(Class.java:438)
    at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:403)
    at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:394)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:393)
    at sun.reflect.MethodAccessorGenerator.generateMethod(MethodAccessorGenerator.java:75)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:53)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.apache.logging.log4j.status.StatusLogger$BoundedQueue.<clinit>(StatusLogger.java)
    at org.apache.logging.log4j.status.StatusLogger.<init>(StatusLogger.java:75)
    at org.apache.logging.log4j.status.StatusLogger.classInitializer$tFvp9PAT(StatusLogger.java:64)
    at org.apache.logging.log4j.status.StatusLogger.<clinit>(StatusLogger.java)
    at org.apache.logging.log4j.LogManager.classInitializer$2uwbj97q(LogManager.java:60)
    at org.apache.logging.log4j.LogManager.<clinit>(LogManager.java)
    at com.ai.cloud.skywalking.sample.util.DubboStart.classInitializer$oOb0838P(DubboStart.java:9)
    at com.ai.cloud.skywalking.sample.util.DubboStart.<clinit>(DubboStart.java)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:260)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:122)
Caused by: java.lang.ClassNotFoundException: sun.reflect.GeneratedMethodAccessor4
    at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 25 more
raphw commented 8 years ago

Since versions 1.4.*, Byte Buddy excludes the bootstrap class loader from instrumentation by default. It seems like you are trying to instrument too many classes including classes within the JVM.

I recommend you to change your strategy for instrumenting. Instead of doing "no-op" transformations like:

if (pluginDefine == null) {
  return builder;
}

you should exclude classes when setting type(exclusivePackageClass()) such that those classes are never triggered. Also, you should not catch exceptions; Byte Buddy catches them and aborts the transformation what is much more performant. You can still log exceptions by registering an AgentBuilder.Listener.

wu-sheng commented 8 years ago

I change my code like this.

new AgentBuilder.Default().type(enhanceClassMatcher(pluginDefineCategory)).transform(new AgentBuilder.Transformer() {
                public DynamicType.Builder<?> transform(DynamicType.Builder<?> builder, TypeDescription typeDescription, ClassLoader classLoader) {
                    AbstractClassEnhancePluginDefine pluginDefine = pluginDefineCategory.findPluginDefine(typeDescription.getTypeName());
                    return pluginDefine.define(typeDescription.getTypeName(), builder);
                }
            }).with(new AgentBuilder.Listener() {
                @Override
                public void onTransformation(TypeDescription typeDescription, ClassLoader classLoader, JavaModule module, DynamicType dynamicType) {

                }

                @Override
                public void onIgnored(TypeDescription typeDescription, ClassLoader classLoader, JavaModule module) {
                    logger.info("ignore to enhance class " + typeDescription.getTypeName());
                }

                @Override
                public void onError(String typeName, ClassLoader classLoader, JavaModule module, Throwable throwable) {
                    logger.error("Failed to enhance class " + typeName, throwable);
                }

                @Override
                public void onComplete(String typeName, ClassLoader classLoader, JavaModule module) {
                    logger.info("enhance class " + typeName + " complete.");
                }
            }).installOn(instrumentation);

And I will test my project today, hope it will work as my wish.

raphw commented 8 years ago

In 1.4.17, the sun.reflect. namespace is now excluded if not specified otherwise. After updating, you should be able to run your agent without any preconfiguration; you should still adapt your mapping. I am closing this issue and but I am happy to reopen it if your problem persists and you can provide more information.