Closed mkoncek closed 2 years ago
Another message:
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f973d542a80, pid=691734, tid=691754
#
# JRE version: OpenJDK Runtime Environment AdoptOpenJDK-dcevm-11.0.11+1-202105021744 (11.0.11+1) (build 11.0.11+1-202105021744)
# Java VM: Dynamic Code Evolution 64-Bit Server VM AdoptOpenJDK-dcevm-11.0.11+1-202105021744 (11.0.11+1-202105021744, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# V [libjvm.so+0xdbea80] Symbol::increment_refcount()+0x0
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" (or dumping to /home/mkoncek/Upstream/classpathless-compiler/xd/xd/core.691734)
#
# An error report file with more information is saved as:
# /home/mkoncek/Upstream/classpathless-compiler/xd/xd/hs_err_pid691734.log
#
# If you would like to submit a bug report, please visit:
# https://github.com/AdoptOpenJDK/openjdk-support/issues
#
Aborted (core dumped)
And you are saying, that it casued update in JRD???
-- Mgr. Jiri Vanek @.***
---------- Původní e-mail ---------- Od: mkoncek @.> Komu: pmikova/java-runtime-decompiler @. github.com> Datum: 6. 12. 2021 18:25:12 Předmět: Re: [pmikova/java-runtime-decompiler] Retransformation no longer works with DCEVM (Issue #228) "
Another message:
A fatal error has been detected by the Java Runtime Environment:
SIGSEGV (0xb) at pc=0x00007f973d542a80, pid=691734, tid=691754
JRE version: OpenJDK Runtime Environment AdoptOpenJDK-dcevm-11.0.11+1- 202105021744 (11.0.11+1) (build 11.0.11+1-202105021744)
Java VM: Dynamic Code Evolution 64-Bit Server VM AdoptOpenJDK-dcevm-11.0.11+ 1-202105021744 (11.0.11+1-202105021744, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
Problematic frame:
V [libjvm.so+0xdbea80] Symbol::increment_refcount()+0x0
Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" (or dumping to /home/mkoncek/Upstream/classpathless-compiler/xd/xd/core.691734)
An error report file with more information is saved as:
/home/mkoncek/Upstream/classpathless-compiler/xd/xd/hs_err_pid691734.log
If you would like to submit a bug report, please visit:
https://github.com/AdoptOpenJDK/openjdk-support/issues (https://github.com/AdoptOpenJDK/openjdk-support/issues)
Aborted (core dumped)
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub (https://github.com/pmikova/java-runtime-decompiler/issues/228#issuecomment-986989971) , or unsubscribe (https://github.com/notifications/unsubscribe-auth/AAWFCSZIRDOJRY4IMM7NEXTUPTWXDANCNFSM5JPFO4IQ) . Triage notifications on the go with GitHub Mobile for iOS (https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675) or Android (https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub) . "
Bad bytecodes?
-- Mgr. Jiri Vanek @.***
---------- Původní e-mail ---------- Od: mkoncek @.> Komu: pmikova/java-runtime-decompiler @. github.com> Datum: 6. 12. 2021 18:23:19 Předmět: [pmikova/java-runtime-decompiler] Retransformation no longer works with DCEVM (Issue #228) "
I am getting various types of errors even when attempting just a no-change reupload of bytecode and the process runs under DCEVM.
Exception: java.lang.UnsatisfiedLinkError thrown from the UncaughtExceptionHandler in thread "Thread-0" Exception: java.lang.UnsatisfiedLinkError thrown from the UncaughtExceptionHandler in thread "main"
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub (https://github.com/pmikova/java-runtime-decompiler/issues/228), or unsubscribe (https://github.com/notifications/unsubscribe-auth/AAWFCSYIFQXLGPSAPTIZO7LUPTWPPANCNFSM5JPFO4IQ) . Triage notifications on the go with GitHub Mobile for iOS (https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675) or Android (https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub) . "
Investigating, but I have no idea.
Attempted java-runtime-decompiler-5.0.beta1
+ CPLC 1.6.1
Confirmed that java-runtime-decompiler-5.0.beta1
+ CPLC 1.5 works in a very simple example (including adding methods)
Latest releases work with disassemblers. And i verified again and it also works when using normal JVM.
I think there is something with BytecodeExtractor...
Good news: Issue has been located and it is indeed something about BytecodeExtractor (namely the second phase). Removing the code makes it work.
Issue is not in JRD, closing.
Maybe not so fast... there is something going on between the interaction of CPLC and JRD.
I can explicitly request classnames from bytecodes but if i call getClassPathListing
after that, it crashes somehow. If I call getClassPathListing
before inspecting bytecodes or don't instepct them at all, it can work.
This needs inspection (I can take a look on that) and possible change in JRD if you want to support DCEVM.
It seems like the agent crashes the remote program: https://github.com/pmikova/java-runtime-decompiler/blob/master/runtime-decompiler/src/main/java/org/jrd/backend/communication/RuntimeCompilerConnector.java#L70
This may turn out to be a bug in DCEVM.
It seems like the current workaround is to conditionally disable bytecode inspection on CPLC side by yet another property. Or add it as another argument to CPLC constructor and settable in JRD.
yes. this must be fixed. I hope you will find a clue, otherwise the git bisect will need to run
-- Mgr. Jiri Vanek @.***
---------- Původní e-mail ---------- Od: mkoncek @.> Komu: pmikova/java-runtime-decompiler @. github.com> Datum: 6. 12. 2021 19:32:46 Předmět: Re: [pmikova/java-runtime-decompiler] Retransformation no longer works with DCEVM (Issue #228) "
Maybe not so fast... there is something going on between the interaction of CPLC and JRD. I can explicitly request classnames from bytecodes but if i call getClassPathListing after that, it crashes somehow. If I call getClassPathListing before inspecting bytecodes or don't instepct them at all, it can work.
This needs inspection (I can take a look on that) and possible change in JRD if you want to support DCEVM.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/pmikova/java-runtime-decompiler/issues/228#issuecomment-987044731) , or unsubscribe (https://github.com/notifications/unsubscribe-auth/AAWFCS5BFZEBCRTOHNLGL33UPT6URANCNFSM5JPFO4IQ) . Triage notifications on the go with GitHub Mobile for iOS (https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675) or Android (https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub) . "
This is not a bug introduced by JRD. This happens because CPLC tries to inspect bytecode and request nested and referenced classes from JRD and then getting classpath listing. For some reason this causes DCEVM to segfault / crash somehow.
yes. but iinspecting bytecode == getBytes agent operation. request classes = = init agent operaton and classpath listing == listClasses agent operation.
So what you are saying, is that cchaining three trivial and safe operations is causing sigsev in dcevm jdk.
Fromt that, my suspicion is going to init, as that was not there when yo u lastt tried that. And that, is leading to a possibel dcevm crasher - init+list classes....
thoutgts?
-- Mgr. Jiri Vanek @.***
---------- Původní e-mail ---------- Od: mkoncek @.> Komu: pmikova/java-runtime-decompiler @. github.com> Datum: 7. 12. 2021 9:29:58 Předmět: Re: [pmikova/java-runtime-decompiler] Retransformation no longer works with DCEVM (Issue #228) "
This is not a bug introduced by JRD. This happens because CPLC tries to inspect bytecode and request nested and referenced classes from JRD and then getting classpath listing. For some reason this causes DCEVM to segfault / crash somehow.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/pmikova/java-runtime-decompiler/issues/228#issuecomment-987685114) , or unsubscribe (https://github.com/notifications/unsubscribe-auth/AAWFCS45F7A2VF5MG7TVNMTUPXAXVANCNFSM5JPFO4IQ) . Triage notifications on the go with GitHub Mobile for iOS (https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675) or Android (https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub) . "
As you are saying: init + listing. Removing init and using only listing seems to work, but you have to manually init classes (which works for some reason).
It would be nice if we could reproduce it somehow.
unittest in jrd should do.
Maybe jsut adding sleep between init and listing?
Or maybe the listing works, because it is in jrd6 launched by different agent session? (not sure here).
Can you try the sleep?
-- Mgr. Jiri Vanek @.***
---------- Původní e-mail ---------- Od: mkoncek @.> Komu: pmikova/java-runtime-decompiler @. github.com> Datum: 7. 12. 2021 12:19:06 Předmět: Re: [pmikova/java-runtime-decompiler] Retransformation no longer works with DCEVM (Issue #228) "
As you are saying: init + listing. Removing init and using only listing seems to work, but you have to manually init classes (which works for some reason).
It would be nice if we could reproduce it somehow.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/pmikova/java-runtime-decompiler/issues/228#issuecomment-987829941) , or unsubscribe (https://github.com/notifications/unsubscribe-auth/AAWFCS4ZN4MTV6LG46OSKETUPXUSJANCNFSM5JPFO4IQ) . Triage notifications on the go with GitHub Mobile for iOS (https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675) or Android (https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub) . "
Sleep (1 s) doesn't do anything.
However, it seems like the problem is caused by getting the bytecode of java.lang.
classes or maybe just initializing them.
I can ignore all the classes from package java.lang.
in bytecode inspection step, then it seems to work.
Also it seems like it is necessary to useHostSystemClasses.
java.lang.BootstrapMethodError: bootstrap method initialization exception
at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:194)
at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
at org.jrd.agent.InstrumentationProvider.findClass(InstrumentationProvider.java:81)
at org.jrd.agent.InstrumentationProvider.findClassBody(InstrumentationProvider.java:70)
at org.jrd.agent.AgentActionWorker.sendByteCode(AgentActionWorker.java:191)
at org.jrd.agent.AgentActionWorker.executeRequest(AgentActionWorker.java:107)
at org.jrd.agent.AgentActionWorker.<init>(AgentActionWorker.java:41)
at org.jrd.agent.ConnectionDelegator.run(ConnectionDelegator.java:77)
Caused by: java.lang.ClassCastException: class java.lang.invoke.MethodType cannot be cast to class java.lang.invoke.MethodHandle (java.lang.invoke.MethodType and java.lang.invoke.MethodHandle are in module java.base of loader 'bootstrap')
at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:99)
... 9 more
Exception in thread "Thread-0" java.lang.BootstrapMethodError: bootstrap method initialization exception
at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:194)
at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
at org.jrd.agent.AgentActionWorker.toError(AgentActionWorker.java:30)
at org.jrd.agent.AgentActionWorker.toError(AgentActionWorker.java:34)
at org.jrd.agent.AgentActionWorker.sendByteCode(AgentActionWorker.java:199)
at org.jrd.agent.AgentActionWorker.executeRequest(AgentActionWorker.java:107)
at org.jrd.agent.AgentActionWorker.<init>(AgentActionWorker.java:41)
at org.jrd.agent.ConnectionDelegator.run(ConnectionDelegator.java:77)
Caused by: java.lang.ClassCastException: class java.lang.invoke.MethodType cannot be cast to class java.lang.invoke.MethodHandle (java.lang.invoke.MethodType and java.lang.invoke.MethodHandle are in module java.base of loader 'bootstrap')
at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:99)
... 9 more
Another what may be cause, is DCEVM 8 xor 11 where jrd and agent and cplc is 11 xor 8
How sleep 100s?
-- Mgr. Jiri Vanek @.***
---------- Původní e-mail ---------- Od: mkoncek @.> Komu: pmikova/java-runtime-decompiler @. github.com> Datum: 7. 12. 2021 13:10:14 Předmět: Re: [pmikova/java-runtime-decompiler] Retransformation no longer works with DCEVM (Issue #228) "
Sleep (1 s) doesn't do anything. However, it seems like the problem is caused by getting the bytecode of java.lang. classes or maybe just initializing them. Also it seems like it is necessary to useHostSystemClasses.
java.lang.BootstrapMethodError: bootstrap method initialization exception
at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:194)
at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307)
at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258)
at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248)
at org.jrd.agent.InstrumentationProvider.findClass(InstrumentationProvider.java:81)
at org.jrd.agent.InstrumentationProvider.findClassBody(InstrumentationProvider.java:70)
at org.jrd.agent.AgentActionWorker.sendByteCode(AgentActionWorker.java:191)
at org.jrd.agent.AgentActionWorker.executeRequest(AgentActionWorker.java:107)
at org.jrd.agent.AgentActionWorker.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/pmikova/java-runtime-decompiler/issues/228#issuecomment-987867010) , or unsubscribe (https://github.com/notifications/unsubscribe-auth/AAWFCSZH5DEWQZML7SFVFDTUPX2R5ANCNFSM5JPFO4IQ) . Triage notifications on the go with GitHub Mobile for iOS (https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675) or Android (https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub) . "
The agent disconnect happens actually before classpath listing, looks like after requesting the bytecode of first non- java.lang.
class having previously obtained bytecodes of a few java.lang
classes.
I thought about dcevm 8 (since dcevm 11 integrated that hotswapagent), but I wasn't able to simply run dcevm 8.
what?!!?!?
-- Mgr. Jiri Vanek @.***
---------- Původní e-mail ---------- Od: mkoncek @.> Komu: pmikova/java-runtime-decompiler @. github.com> Datum: 7. 12. 2021 15:01:25 Předmět: Re: [pmikova/java-runtime-decompiler] Retransformation no longer works with DCEVM (Issue #228) "
The agent disconnect happens actually before classpath listing, looks like after requesting the bytecode of first non- java.lang. class having previously obtained bytecodes of a few java.lang classes. I thought about dcevm 8 (since dcevm 11 integrated that hotswapagent), but I wasn't able to simply run dcevm 8.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/pmikova/java-runtime-decompiler/issues/228#issuecomment-987952796) , or unsubscribe (https://github.com/notifications/unsubscribe-auth/AAWFCS5O2FLWMG32YJXLXZ3UPYHSZANCNFSM5JPFO4IQ) . Triage notifications on the go with GitHub Mobile for iOS (https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675) or Android (https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub) . "
I really don't know... This seems to be a very simple, non-invasive workaround that doesn't stop anything other from working. https://github.com/mkoncek/classpathless-compiler/commit/5f98b6428385c89391cf38881e0b2d4bc39d6a3a
Actually, just blocking java.lang.Object
seems to do the trick... Don't ask me why.
Ok, this looks like a race condition in DCEVM and can be easily reproduced by:
System.out.println(classesProvider.getClass(new ClassIdentifier("java.lang.Object")).size());
I am getting different results, sometimes the program crashes (when the size is 1).
Other times just the agent crashes.
Blocking how? Sound like reasonable way to do.
-- Mgr. Jiri Vanek @.***
---------- Původní e-mail ---------- Od: mkoncek @.> Komu: pmikova/java-runtime-decompiler @. github.com> Datum: 7. 12. 2021 15:29:51 Předmět: Re: [pmikova/java-runtime-decompiler] Retransformation no longer works with DCEVM (Issue #228) "
Actually, just blocking java.lang.Object seems to do the trick... Don't ask me why.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/pmikova/java-runtime-decompiler/issues/228#issuecomment-987977989) , or unsubscribe (https://github.com/notifications/unsubscribe-auth/AAWFCS6Y6YI376QIS642X5TUPYK5PANCNFSM5JPFO4IQ) . Triage notifications on the go with GitHub Mobile for iOS (https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675) or Android (https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub) . "
There are two instances when that bytecode is requested: in the pre-compile phase of bytecode inspection (because just about anything references Object
) and when useHostSystemClasses is disabled.
Special-casing the 1st case is simple and we don't lose any information (the bytecode is requested just to find out if the class is nested within another class, which Object
never is).
Special-casing the 2nd case would require some code delving and not sure if it is worth it. By using useHostSystemClasses(true)
(which is the default) we do not request the bytecode of java.lang.Object
.
no idea what you are speaking abou t:) lets continue in monday , otherwise you wil burry yoursel fin tthis very intereting corenr case
-- Mgr. Jiri Vanek @.***
---------- Původní e-mail ---------- Od: mkoncek @.> Komu: pmikova/java-runtime-decompiler @. github.com> Datum: 7. 12. 2021 16:15:03 Předmět: Re: [pmikova/java-runtime-decompiler] Retransformation no longer works with DCEVM (Issue #228) "
There are two instances when that bytecode is requested: in the pre-compile phase of bytecode inspection (because just about anything references Object) and when useHostSystemClasses is disabled. special-casing the 1st case is simple and we don't lose any information (the bytecode is requested just to find out if the class is nested within another class, which Object never is) special-casing the 2nd case would require some code delving and not sure if it is worth it
— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/pmikova/java-runtime-decompiler/issues/228#issuecomment-988019614) , or unsubscribe (https://github.com/notifications/unsubscribe-auth/AAWFCS6OCOKH3J3YHW2E4GTUPYQG7ANCNFSM5JPFO4IQ) . Triage notifications on the go with GitHub Mobile for iOS (https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675) or Android (https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub) . "
The workaround wasn't difficult: https://github.com/mkoncek/classpathless-compiler/commit/5a93a7825fff510ff99e6c9db3ea84d2dd05c835
With this modification, it also works when useHostSystemClasses(false)
, but in this case it always loads the host java.lang.Object
which is probably not always desirable. Maybe should make the behaviour selectable from JRD.
With this and the previous commits, DCEVM works both with and without useHostSystemClasses
.
gosh. what a weird thing. Thanx for it. If you allow to be optional, would be great. TY!
-- Mgr. Jiri Vanek @.***
---------- Původní e-mail ---------- Od: mkoncek @.> Komu: pmikova/java-runtime-decompiler @. github.com> Datum: 8. 12. 2021 13:49:06 Předmět: Re: [pmikova/java-runtime-decompiler] Retransformation no longer works with DCEVM (Issue #228) "
The workaround wasn't difficult: @.*** (https://github.com/mkoncek/classpathless-compiler/commit/5a93a7825fff510ff99e6c9db3ea84d2dd05c835)
With this modification, it also works when useHostSystemClasses(false), but in this case it always loads the host java.lang.Object which is probably not always desirable. Maybe should make the behaviour selectable from JRD.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub (https://github.com/pmikova/java-runtime-decompiler/issues/228#issuecomment-988786726) , or unsubscribe (https://github.com/notifications/unsubscribe-auth/AAWFCS2FHFOWIVO2EJWSX7TUP5H3VANCNFSM5JPFO4IQ) . Triage notifications on the go with GitHub Mobile for iOS (https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675) or Android (https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub) . "
Addressed: https://github.com/mkoncek/classpathless-compiler/commit/109bba043f6ef7ed809000975a64669b43425fba One more selectable checkbox for JRD :)
swithc will be added
Hello! The switch was added into sessionAgent branch. Do you mind to try it out please? Or how can I reproduce it?
Checked and it seems to work as intended. Let me reexplain:
useHostJavaLangObject
is like useHostSystemClasses
but for java.lang.Object
only.
When useHostSystemClasses
is true, then the other option is implicitly true (its value does not matter).
Only when useHostSystemClasses
is false, the value of the other option matters.
When both are false, then you will be getting segfaults and other funny stuff on processes running under DCEVM VM.
fixed for 6.0
extended in 7.0
I am getting various types of errors even when attempting just a no-change reupload of bytecode and the process runs under DCEVM.