search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.28k stars 721 forks source link

jsr292 performance investigation #4837

Open cathyzhyi opened 5 years ago

cathyzhyi commented 5 years ago

OpenJ9 does not perform very well with programs heavily using jsr292. This issue is for tracking investigation into the performance issue. The ideas are still in experimental state and I will update PR link once they are ready for delivery.

  1. Swapping targeted inlining with normal inlining Targeted inlining was done before normal inlining and won't do any thing if the jsr292 methdos were called from a inlined callee. Swapping to do normal inlining first so that jsr292 methods called from callees can be inlined as well. This can result in leaf java methods not being inlined so we will need to check how does it work in practice.

  2. add small recongnized jsr292 methods as inlineable target for both targeted inlining and normal inlining: java_lang_invoke_MethodHandle_type
    java_lang_invoke_ArgumentMoverHandle_extra_L java_lang_invoke_BruteArgumentMoverHandle_extra_L

  3. MutableCallSite target inlining. There are several problems with mutable callsite a. MutableCallSiteTargetGuard exists already for inlining mutable callsite and if a method handle is got from MutableCallSite.getTarget() or MutableCallSite.target the guard is created and the methodhandle chain can be inlined. HCR guard breaks the pattern by storing the result into a temp when inlining MutableCallSite.getTarget. Need to find a way to keep the pattern so that we can inline mutable callsite target. One cleaner way would be changing the MutableCallSite.target field to package access so that we don't need to access the field through function call. b. once the target is inlined, propagate known obj info down the call chain.

  4. there is a condition to only inline jsr292 methods at hot level. change it to inline at warm as well. https://github.com/eclipse/openj9/blob/ad68fc5e4ef89bdea9f81df7ce2320c56988c38b/runtime/compiler/env/VMJ9.cpp#L3138-L3142

@vijaysun-omr @DanHeidinga @andrewcraik

cathyzhyi commented 5 years ago

Tried out prototype changes for item 1-4 in description on JRuby benchmarks from https://github.com/headius/bench2018. Posting the results in the table below. Benchmark names are bold and some benchmarks have subbenchmarks. The relative numbers with % are relative to baseline in terms of performance. withchange includes item 2-4. withchangeinlineathot includes item 1-3. withchangeswap includes item 1-4 especially adding swapping normal inlining and targeted inlining compared to withchange. Swapping shows improvement in most benchmarks but hurts red_black and mandelbrot because leaf methods are not inlined after targeted inlining. hash has 5 subbenchmarks and seems to have big fluctuation with the new changes and need further investigation.

red_black
LOWER is better
config hotspot baseline withchange withchangeinlineathot withchangeswap
mean 0.26 5.48 1.49 (+268%) 3.29 (+67%) 2.35 (+133%)
std 0.01 1.17 0.12 0.95 0.05
aref
HIGHER is better
Instantiation
config hotspot baseline withchange withchangeinlineathot withchangeswap
mean 102.77 7.87 32.1 (+308%) 31.59 (+301%) 31.49 (+300%)
std 1.01 2.84 0.3 0.69 0.33
hash
HIGHER is better
bm_hash_small4
config hotspot baseline withchange withchangeinlineathot withchangeswap
mean 306.44 141.28 127.58 (-10%) 142.65 (+1%) 146.74 (+4%)
std 2.71 4.51 21.0 4.27 4.71
bm_hash_long
config hotspot baseline withchange withchangeinlineathot withchangeswap
mean 90.7 68.84 73.31 (+6%) 63.01 (-8%) 68.46 (-1%)
std 0.4 2.32 1.27 7.93 4.65
bm_hash_small8
config hotspot baseline withchange withchangeinlineathot withchangeswap
mean 187.26 100.64 89.12 (-11%) 100.94 (+0%) 106.6 (+6%)
std 1.74 3.44 17.68 4.57 2.68
bm_bighash
config hotspot baseline withchange withchangeinlineathot withchangeswap
mean 245.55 112.13 107.82 (-4%) 109.48 (-2%) 155.15 (+38%)
std 1.72 2.63 6.12 12.23 2.3
bm_hash_small2
config hotspot baseline withchange withchangeinlineathot withchangeswap
mean 458.39 189.2 170.98 (-10%) 184.1 (-3%) 196.88 (+4%)
std 3.65 3.22 21.45 4.4 14.24
weirdsort
HIGHER is better
weird
config hotspot baseline withchange withchangeinlineathot withchangeswap
mean 38.61 8.58 9.7 (+13%) 9.7 (+13%) 12.4 (+45%)
std 7.77 0.27 0.21 0.06 0.2
normal
config hotspot baseline withchange withchangeinlineathot withchangeswap
mean 135.41 4.75 19.22 (+305%) 14.28 (+201%) 31.08 (+554%)
std 3.08 0.06 1.34 0.82 0.9
mandelbrot
LOWER is better
config hotspot baseline withchange withchangeinlineathot withchangeswap
mean 1.21 16.73 2.78 (+502%) 14.56 (+15%) 3.35 (+399%)
std 0.01 0.32 0.03 0.13 0.03
kwargs
LOWER is better
config hotspot baseline withchange withchangeinlineathot withchangeswap
mean 1.51 2.99 2.14 (+40%) 2.16 (+38%) 1.98 (+51%)
std 0.01 0.42 0.18 0.29 0.03
cathyzhyi commented 5 years ago

The above work improves the performance to some degree but there still splenty more to do. Looked more into benchmarks performs espically poorly red_back:     1432 4.80% java/lang/invoke/MutableCallSiteDynamicInvokerHandle.invokeExact_thunkArchetype_X(Ljava/lang/Object;Ljava/lang/Object;Ljava     2438 8.16% java/lang/invoke/MutableCallSiteDynamicInvokerHandle.invokeExact_thunkArchetype_X(Ljava/lang/Object;Ljava/lang/Object;Ljava     2561 8.58% home/yizhang/methodhandle/jruby/bench2018/bench_red_black.RUBY$method$insert_helper$27(Lorg/jruby/runtime/ThreadContext;Lor     9368 31.37% VM_BytecodeInterpreter::run     quite a lot tickes in interpreter, mutable callsite are not inlined. These ticks only shows up when swapping targeted and normal inlining. Experimental changes to inline leaf java methods fixed this but hurt the other benchmarks. Need to investigate to try to find a more fine tuned heuristic. aref     Lhome/yizhang/methodhandle/jruby/bench2018/bench_aref;::RUBY$block$\=|home|yizhang|methodhandle|jruby|bench2018|bench_aref\,rb$1     The hottest method takes 80% ticks in both j9 and hotspot. Will look at the log. weirdsort normal bm_hash_small2     Ljava/lang/invoke/LambdaForm$BMH/1338841523;::reinvoke (hottest in j9)     home/yizhang/methodhandle/jruby/bench2018/bench_hash_small2only.RUBY$block$     home/yizhang/methodhandle/jruby/bench2018/bench_weird_sort_normalonly.RUBY$block$\ (hottest for us)     in both benchmarks, the hottest method for j9 is JRuby method but hotspot is LambdaForm$BMH. Probably need to look into LambdaForm

In summary, next steps are:

DanHeidinga commented 5 years ago

OpenJ9 doesn't use LambdaForms - that's OpenJDK's implementation of MethodHandles. Where we build up a chain of MH-subclasses to represent the behaviour, they generate classes from the LambdaForms to implement the MH behaviour.

cathyzhyi commented 5 years ago

@DanHeidinga Thanks for the information! I will just investigate the openj9 tracing log then.

avermeer commented 5 years ago

Hello, I submitted #5371. I volunteer to retry its associated bench test case with any OpenJ9 binaries (on Linux or Windows) that you think might help improve the performances for this - even if the binaries aren't yet on nightly builds (in which case i'll need an URL to download them).

Running the test take about 1 hour, but I'm patient - especially if that can help.

cathyzhyi commented 5 years ago

@avermeer Thanks a lot for offering to help. We've tried the Nashorn benchmark with the improvements done for JRudy but they don't seem to help much. We think it should be because of the same reason mentioned in https://github.com/eclipse/openj9/issues/4837#issuecomment-472105631

Swapping shows improvement in most benchmarks but hurts red_black and mandelbrot because leaf methods are not inlined after targeted inlining.

We are working on it and will let you known once we have the fix for you to test out. Thanks!

cathyzhyi commented 5 years ago

As found in https://github.com/eclipse/openj9/issues/4837#issuecomment-472105631, swapping inlining and targeted inlining causes regression on some JRuby benchmarks because of leaf java method at the end of methodhandle chain not inlined by targeted inlining. After changing inliner to inline leaf method the performance is improved a little bit. The regression caused by swapping inlining in red_black , aref , weird_sort are fixed but the improvement is not as large as we expected.

nonInlineLeaf are previous changes without the inline through methodhandle chain to leaf method in normal inlining and inlineLeaf includes both previous changes and this change.

red_black
LOWER is better
config hotspot baseline inlineLeaf nonInlineLeaf
mean 0.27 6.02 1.5 (+301%) 2.42 (+149%)
std 4% 29% 3% 2%
aref
HIGHER is better
Instantiation
config hotspot baseline inlineLeaf nonInlineLeaf
mean 117.79 5.63 28.85 (+412%) 25.3 (+349%)
std 2% 7% 9% 7%
hash
HIGHER is better
bm_hash_small4
config hotspot baseline inlineLeaf nonInlineLeaf
mean 345.19 169.91 168.21 (-1%) 162.86 (-4%)
std 1% 4% 6% 9%
bm_hash_long
config hotspot baseline inlineLeaf nonInlineLeaf
mean 99.72 70.18 80.02 (+14%) 79.04 (+13%)
std 1% 7% 3% 6%
bm_hash_small8
config hotspot baseline inlineLeaf nonInlineLeaf
mean 212.5 118.46 121.04 (+2%) 119.33 (+1%)
std 1% 3% 7% 4%
bm_bighash
config hotspot baseline inlineLeaf nonInlineLeaf
mean 273.52 139.73 186.73 (+34%) 171.4 (+23%)
std 1% 2% 7% 6%
bm_hash_small2
config hotspot baseline inlineLeaf nonInlineLeaf
mean 501.0 226.04 219.19 (-3%) 227.49 (+1%)
std 5% 4% 6% 6%
mandelbrot
LOWER is better
config hotspot baseline inlineLeaf nonInlineLeaf
mean 1.22 18.0 4.93 (+265%) 4.57 (+294%)
std 1% 3% 3% 9%
kwargs
LOWER is better
config hotspot baseline inlineLeaf nonInlineLeaf
mean 1.6 3.88 2.71 (+43%) 2.87 (+35%)
std 1% 16% 9% 10%
weird_sort
HIGHER is better
weird
config hotspot baseline inlineLeaf nonInlineLeaf
mean 37.96 6.83 10.62 (+55%) 8.93 (+31%)
std 22% 4% 9% 8%
normal
config hotspot baseline inlineLeaf nonInlineLeaf
mean 147.97 5.12 28.11 (+449%) 22.38 (+337%)
std 5% 4% 11% 5%

nashorn still doesn't perform well. We found the hottest method's compilation is aborted because of inliner using too much memory. Trying to reduce footprint of inliner.

vijaysun-omr commented 5 years ago

Thanks, these results show inlineLeaf (which I think is your new inliner config) doing better than other OpenJ9 configs you show in multiple cases with the odd case that is regressed but on the whole inlineLeaf seems like a step in the right direction.

I would like to know if you feel there is some other major issue (apart from aborted compilation because of too much memory) that is responsible for the large delta vs Hotspot in the above test cases.

cathyzhyi commented 5 years ago

We see quite a lot of small issues resulting in the code to be suboptimal, like known object info not propagated correctly, vp didn't fold some obvious cases, some unnecessary indirect load. Some benchmarks like mandelbrot still have inliner issues. Overall, I think inlining the right things should improve the performance by a lot but is not enough to close the gap. Haven't found other major issues yet.

avermeer commented 5 years ago

Hello,

Even if improvements previously quoted are not yet as good as expected to beat HotSpot's, would it be possible to have an OpenJ9 version (nightly build or other way to get binaries) including these improvements so that I could get my real applications the performance benefit?

Kind regards, Alexandre Vermeerbergen

andrewcraik commented 5 years ago

@avermeer we can certainly try to make a branch available - obvious caveats that it won't have gone through full testing etc and we know the new scheme can cause regressions in some extreme circumstances (which will be fixed before the code is delivered). @cathyzhyi @liqunl and I are all looking at the method handle performance to continue the performance improvements.

avermeer commented 5 years ago

@andrewcraik thanks for your answer, I am still interested in even WIP (Work In Progress) version of this issues' fix, if that can help. Also interested if you have any update information to share here...

Kind regards, Alexandre Vermeerbergen

andrewcraik commented 5 years ago

@avermeer - thanks for reaching out. It has been a crazy week which is why we haven't been posting as much as we would like to. @liqunl and @cathyzhyi are working fulltime on this with a bit of help fromme when I get time. The current 'best' prototype is available from @liqunl forks of OpenJ9 (https://github.com/liqunl/openj9/tree/bestsofar) and OMR (https://github.com/liqunl/openj9-omr/tree/bestsofar). The bulk of the changes in this set of branches so far fixes a number of limitations in the inliner. The current inliner in HEAD OpenJ9 can only handle MethodHandle inlining in the outter most compiled method (as discussed previously) and the sum of the changes in these bestsofar branches removes that limitation. We have had good performance results on the smaller tests of jRuby benchmark suite (some of them improve by more than 4x). That being said, we are aware of a significant limitation in this prototype - the techniques being used inside the inliner to follow all of the MethodHandle chains is too expensive (we are peeking the handle bodies which means generating compiler IL to figure out what the next handle is) and this is causing the compiler to run out of memory on some of the larger JSR292 uses in the Nashorn benchmark suite. @cathyzhyi is working on an improvement to the prototype so that we don't need to do the expensive peeking operation to find the things we want to inline (this improvement is well underway, but not quite done yet). In parallel @liqunl is looking at the Java implementation of method handles looking for ways we can make them more efficient and we'll be having a wider discussion on that with the folks here at OpenJ9 once she has finished her analysis.

So sorry for the lack of communication, but a lot of work is happening. If you want to try out the prototype please do so - let us know how you go and we'd be happy to help you gather stats to help inform the work we are doing. That being said we expect to land further significant improvements into the best-so-far branches soon (I'm hoping within a week but @liqunl and @cathyzhyi might disagree :) ). If these further improvements solve the Nashorn benchmark problems then we will look to try and deliver this prototype to HEAD and then continue the investigation.

avermeer commented 5 years ago

Hello @andrewcraik

Thank you very much for this detailed answer.

I used to build the complete AdoptOpenJDK with OpenJ9 distribution from its source once, but here I need to figure out how to take advantage of the OpenJ9 (https://github.com/liqunl/openj9/tree/bestsofar) and OMR (https://github.com/liqunl/openj9-omr/tree/bestsofar) repositories you mentionned.

I cloned https://github.com/liqunl/openj9/tree/bestsofar, but I don't know how to build binaries from this source tree (I'm on Linux, to keep things simple - even if eventually I need a fix on Both Linux & Windows).

Sorry if my question sounds a bit trivial, but how can I build OpenJ9 & OMR to test currently fixes?

Kind regards, Alexandre

andrewcraik commented 5 years ago

@fjeremic could you see if you can help with the build question?

@avermeer a further update that @cathyzhyi and @liqunl have push some additional changes to the best-so-far branches to with the Nashorn problems. We are still in the process of verifying the performance but think it should be better. I'll post again when we have the data, but just to let you know that there are fresh changes landed to help improve things.

pshipton commented 5 years ago

Referring to the OpenJ9 build instructions https://www.eclipse.org/openj9/oj9_build.html

After the bash ./get_source.sh step there will be openj9 and omr git clone directories. These directories need to be updated, using git commands, to the bestsofar branches. Basically with git you add a remote to liqunl's fork, and checkout those branches.

Then continue with the build instructions.

pshipton commented 5 years ago

or just modify the bash ./get_source.sh step to clone the bestsofar branches directly.

pshipton commented 5 years ago 
Usage: ./get_source.sh [-h|--help] [... other j9 options] [-parallel=<true|false>] [--openssl-version=<openssl version to download>]
where:
  -h|--help         print this help, then exit

  -openj9-repo      the OpenJ9 repository url: https://github.com/eclipse/openj9.git
                    or git@github.com:<namespace>/openj9.git
  -openj9-branch    the OpenJ9 git branch: master
  -openj9-sha       a commit SHA for the OpenJ9 repository
  -omr-repo         the OpenJ9/omr repository url: https://github.com/eclipse/openj9-omr.git
                    or git@github.com:<namespace>/openj9-omr.git
  -omr-branch       the OpenJ9/omr git branch: openj9
  -omr-sha           a commit SHA for the omr repository
  -parallel         (boolean) if 'true' then the clone j9 repository commands run in parallel, default is false
  --openssl-version Specify the version of OpenSSL source to download
fjeremic commented 5 years ago

@avermeer I've build a personal build for you [1]:

fjeremic@FILIP-IBM:/tmp$ j2sdk-image/bin/java -version
openjdk version "1.8.0_202-internal"
OpenJDK Runtime Environment (build 1.8.0_202-internal-jenkins_2019_05_09_11_29-b00)
Eclipse OpenJ9 VM (build bestsofar-41ab89d81, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20190509_8 (JIT enabled, AOT enabled)
OpenJ9   - 41ab89d81
OMR      - 5ade255f
JCL      - 6186c40a2e based on jdk8u212-b04)

It has the same SHAs as the two referenced branches. You can find the direct Linux x86-64 download link at the top of the page (OpenJ9-JDK8-x86-64_linux-20190509-152642.tar.gz). Alternatively you can build the thing yourself using instructions outlined by Peter above.

[1] https://ci.eclipse.org/openj9/view/all/job/Build_JDK8_x86-64_linux_Personal/8/

cathyzhyi commented 5 years ago

@avermeer when running with the new build most of the changes kicks in by default but there is one that needs -Xjit:PropagateWithOutPeeking to enable.

avermeer commented 5 years ago

@fjeremic thank you very much, I have downloaded the "personal build" of OpenJDK with OpenJ9.

I see it is a based on Java 8 update 202, so not exactly latest update 212 but I guess that this won't make a big difference for the JSR292 performances.

I am going to use this binary to check if not only the benchmark submitted in #5371 but also our various apps' performance take benefit of this WIP OpenJ9.

And @cathyzhyi thank you very much for the note about -Xjit:PropagateWithOutPeeking

Kind regards, Alexandre

cathyzhyi commented 5 years ago

We've found and fixed various problems related to JSR292 and the investigation is still ongoing. Here are the latest results from a best so far build for JRuby benchmarks and results for nashorn are posted in another issue https://github.com/eclipse/openj9/issues/5371

red_black
LOWER is better
config hotspot baseline bestsofar
mean 0.27 4.25 1.04 (+309%)
std 0% 28% 4%
aref
HIGHER is better
Instantiation
config hotspot baseline bestsofar
mean 119.34 8.14 25.69 (+216%)
std 1% 50% 9%
hash
HIGHER is better
bm_hash_small4
config hotspot baseline bestsofar
mean 356.32 161.36 175.79 (+9%)
std 3% 6% 3%
bm_hash_long
config hotspot baseline bestsofar
mean 99.67 70.44 86.91 (+23%)
std 1% 2% 5%
bm_hash_small8
config hotspot baseline bestsofar
mean 218.34 114.92 129.39 (+13%)
std 3% 5% 6%
bm_bighash
config hotspot baseline bestsofar
mean 271.12 137.93 199.0 (+44%)
std 1% 3% 5%
bm_hash_small2
config hotspot baseline bestsofar
mean 518.6 216.95 239.73 (+11%)
std 1% 5% 6%
mandelbrot
mandelbrot
LOWER is better
config hotspot baseline bestsofar
mean 1.18 18.65 2.97 (+528%)
std 2% 4% 9%
kwargs
LOWER is better
config hotspot baseline bestsofar
mean 1.61 3.7 2.64 (+40%)
std 1% 15% 11%
weird_sort
HIGHER is better
weird
config hotspot baseline bestsofar
mean 41.9 6.62 8.81 (+33%)
std 3% 2% 4%
normal
config hotspot baseline bestsofar
mean 147.98 5.05 22.39 (+343%)
std 3% 6% 3%
avermeer commented 5 years ago

Hello @cathyzhyi ,

Any chance we could use nightly build to take advantage of these improvements?

Kind regards, Alexandre

andrewcraik commented 5 years ago

@avermeer @liqunl and @cathyzhyi are working to get their code delivered - until they are merged it won't be in the nightly build. Once the pull requests start to open (should be in the next roughly week) they will be referenced here so you can follow along. Getting the changes into the nightly build requires they reach a level of polish where they won't cause problems for users in general since hiding all the changes under an option would be quite difficult given how extensive and pervasive the improvements have been in the inliner infrastructure.

The focus is now on getting the changes in the best so far branch merged. Once the pull requests are opening continuing to look at the remaining performance issues for the jRuby suite will be the next thing on the list. So far we have looked in detail at the weird sort - the performance delta there seems to be due to one missed opportunity to stack allocate an object and a set of 0 stores not being eliminated from a hot loop. We've also looked at the bm_hash_small2 which seems to be spending a lot of time allocating objects - there might be a GC tuning or something for this, but we're still working to understand it. We will look at the others soon and we will address the problems we find, but only once we have gotten the improvements so far delivered since they are a big improvement for a number benchmarks.

avermeer commented 5 years ago

@andrewcraik thank you very much for providing such detailed answer. Keep up the good work!

avermeer commented 5 years ago

Hello OpenJ9 developers,

Looks like couple of PR related to this issue have been recently merged. Does it means that with recent AdoptOpenJDK with OpenJ9 nightly build some improvements with Nashhorn can be tested?

Kind regards, Alexandre

liqunl commented 5 years ago

Hi @avermeer Yes, the latest nightly build is compiled with openj9 sha ffeaf288e, a few changes are already in. However, you may not see significant performance improvement as the changes in that build is only a small portion of the changes we have.

avermeer commented 5 years ago

Hello @liqunl ,

Thank you very much for you answer. When do you think an AdoptOpenJDK nightly build could be available with the bulk of the changes for JSR292 performances?

Kind regards, Alexandre Vermeerbergen

avermeer commented 5 years ago

Hello,

Are there news of this issue? Anything that's going to be part of AdoptOpenJDK OpenJ9 July release? Or even nightly builds?

Kind regards, Alexandre

cathyzhyi commented 5 years ago

@avermeer about 60% of the PRs have got into the latest nightly build already and we are still working on the rest of PRs.

avermeer commented 5 years ago

Hello @cathyzhyi ,

Thank you very much, we're are going to redo our tests with this updated nightly build.

Kind regards, Alexandre

Pr0methean commented 4 years ago

This bug hasn't been updated in months. Is there still a performance problem? A way to test in pure Java would be with JEP280 string concatenation

avermeer commented 4 years ago

Hello,

I'll try to redo my scenario and let you know ASAP the outcome with latest OpenJ9 binaires available on Adoptopenjdk.com site.

Kind regards, Alexandre

liqunl commented 4 years ago

@Pr0methean All improvements we developed have been delivered and you should see a decrease in the performance gap between openj9 and openjdk. Currently we have stopped working on this issue because we're going to replace our MethodHandle implementation with openjdk's. See #7352. We'll have another round of performance tuning and improvement once the functionality is achieved with the replacement.

avermeer commented 4 years ago

@liqunl when was latest iteration on this incident? is it in openj9-0.18.1, or should I use a recent nighty build instead ?

liqunl commented 4 years ago

@avermeer Yes, all of our improvements have been released.

avermeer commented 4 years ago

Hello @liqunl ,

Well bad news, with openj9-0.18.1 (I used binaries of officiallly released AdopOpenJDK 11.0.6 for Linux x64 on CentOS 7.6), my "bench" just crashes, see traces:

[root@ip-172-31-23-102 mnt]# ./jdk11-openj9-0.18.1/bin/java -jar target/benchmarks.jar 2>&1 | tee /jdk11-openj9-0.18.1.log
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.openjdk.jmh.util.Utils (file:/mnt/target/benchmarks.jar) to field java.io.PrintStream.charOut
WARNING: Please consider reporting this to the maintainers of org.openjdk.jmh.util.Utils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
# JMH version: 1.21
# VM version: JDK 11.0.6, Eclipse OpenJ9 VM, openj9-0.18.1
# *** WARNING: This VM is not supported by JMH. The produced benchmark data can be completely wrong.
# VM invoker: /mnt/jdk11-openj9-0.18.1/bin/java
# VM options: -Xoptionsfile=/mnt/jdk11-openj9-0.18.1/lib/options.default -Xlockword:mode=default,noLockword=java/lang/String,noLockword=java/util/MapEntry,noLockword=java/util/HashMap$Entry,noLockword=org/apache/harmony/luni/util/ModifiedMap$Entry,noLockword=java/util/Hashtable$Entry,noLockword=java/lang/invoke/MethodType,noLockword=java/lang/invoke/MethodHandle,noLockword=java/lang/invoke/CollectHandle,noLockword=java/lang/invoke/ConstructorHandle,noLockword=java/lang/invoke/ConvertHandle,noLockword=java/lang/invoke/ArgumentConversionHandle,noLockword=java/lang/invoke/AsTypeHandle,noLockword=java/lang/invoke/ExplicitCastHandle,noLockword=java/lang/invoke/FilterReturnHandle,noLockword=java/lang/invoke/DirectHandle,noLockword=java/lang/invoke/ReceiverBoundHandle,noLockword=java/lang/invoke/DynamicInvokerHandle,noLockword=java/lang/invoke/FieldHandle,noLockword=java/lang/invoke/FieldGetterHandle,noLockword=java/lang/invoke/FieldSetterHandle,noLockword=java/lang/invoke/StaticFieldGetterHandle,noLockword=java/lang/invoke/StaticFieldSetterHandle,noLockword=java/lang/invoke/IndirectHandle,noLockword=java/lang/invoke/InterfaceHandle,noLockword=java/lang/invoke/VirtualHandle,noLockword=java/lang/invoke/PrimitiveHandle,noLockword=java/lang/invoke/InvokeExactHandle,noLockword=java/lang/invoke/InvokeGenericHandle,noLockword=java/lang/invoke/VarargsCollectorHandle,noLockword=java/lang/invoke/ThunkTuple -Xjcl:jclse29 -Dcom.ibm.oti.vm.bootstrap.library.path=/mnt/jdk11-openj9-0.18.1/lib/compressedrefs:/mnt/jdk11-openj9-0.18.1/lib -Dsun.boot.library.path=/mnt/jdk11-openj9-0.18.1/lib/compressedrefs:/mnt/jdk11-openj9-0.18.1/lib -Djava.library.path=/mnt/jdk11-openj9-0.18.1/lib/compressedrefs:/mnt/jdk11-openj9-0.18.1/lib:/usr/lib64:/usr/lib -Djava.home=/mnt/jdk11-openj9-0.18.1 -Duser.dir=/mnt -Djava.class.path=target/benchmarks.jar -Dsun.java.command=target/benchmarks.jar -Dsun.java.launcher=SUN_STANDARD -Dsun.java.launcher.pid=27614
# Warmup: 5 iterations, 10 s each
# Measurement: 5 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.test.MyBenchmark.closureTest

WARNING: Not a HotSpot compiler command compatible VM ("Eclipse OpenJ9 VM-11.0.6"), compilerHints are disabled.
# Run progress: 0.00% complete, ETA 00:50:00
# Fork: 1 of 5
# Warmup Iteration   1: Warning: Nashorn engine is planned to be removed from a future JDK release
Assertion failed at /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-x64-openj9/workspace/build/src/build/linux-x86_64-normal-server-release/vm/compiler/../compiler/optimizer/InterpreterEmulator.cpp:208: 0
VMState: 0x000501ff
        unexpected bytecode in thunk archetype 0x7f13499d9200 at bcIndex 40 JBlload2 (33)

compiling jdk/nashorn/internal/scripts/Script$Recompilation$161$654$\^eval\_.L:5#setup#L:23#L:26(Ljdk/nashorn/internal/runtime/ScriptFunction;Ljava/lang/Object;)Ljava/lang/Object; at level: warm
#0: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x8aa205) [0x7f1398b3b205]
#1: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x8b6380) [0x7f1398b47380]
#2: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x5783bb) [0x7f13988093bb]
#3: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x5785ad) [0x7f13988095ad]
#4: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x26410c) [0x7f13984f510c]
#5: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x267d82) [0x7f13984f8d82]
#6: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x256120) [0x7f13984e7120]
#7: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x256f4f) [0x7f13984e7f4f]
#8: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x256f4f) [0x7f13984e7f4f]
#9: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x256f4f) [0x7f13984e7f4f]
#10: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x256f4f) [0x7f13984e7f4f]
#11: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x258682) [0x7f13984e9682]
#12: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x23a042) [0x7f13984cb042]
#13: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x245ba9) [0x7f13984d6ba9]
#14: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x24d70b) [0x7f13984de70b]
#15: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x666c6e) [0x7f13988f7c6e]
#16: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x245036) [0x7f13984d6036]
#17: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x728427) [0x7f13989b9427]
#18: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x729d8b) [0x7f13989bad8b]
#19: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x51a5fd) [0x7f13987ab5fd]
#20: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x184086) [0x7f1398415086]
#21: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x184ec1) [0x7f1398415ec1]
#22: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9prt29.so(+0x20df3) [0x7f13a456fdf3]
#23: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x186b95) [0x7f1398417b95]
#24: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x18712e) [0x7f139841812e]
#25: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x182d63) [0x7f1398413d63]
#26: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x1831b2) [0x7f13984141b2]
#27: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x18325a) [0x7f139841425a]
#28: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9prt29.so(+0x20df3) [0x7f13a456fdf3]
#29: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x1836b4) [0x7f13984146b4]

JVMDUMP039I Processing dump event "abort", detail "" at 2020/03/08 09:20:34 - please wait.
JVMDUMP032I JVM requested System dump using '/mnt/core.20200308.092034.27648.0001.dmp' in response to an event
Assertion failed at /home/jenkins/workspace/build-scripts/jobs/jdk11u/jdk11u-linux-x64-openj9/workspace/build/src/build/linux-x86_64-normal-server-release/vm/compiler/../compiler/optimizer/InterpreterEmulator.cpp:208: 0
VMState: 0x000501ff
        unexpected bytecode in thunk archetype 0x7f13488dcd50 at bcIndex 40 JBlload2 (33)

compiling jdk/nashorn/internal/scripts/Script$Recompilation$11$589$\^eval\_.L:5#setup#L:23(Ljava/lang/Object;)Ljava/lang/Object; at level: warm
#0: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x8aa205) [0x7f1398b3b205]
#1: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x8b6380) [0x7f1398b47380]
#2: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x5783bb) [0x7f13988093bb]
#3: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x5785ad) [0x7f13988095ad]
#4: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x26410c) [0x7f13984f510c]
#5: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x267d82) [0x7f13984f8d82]
#6: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x256120) [0x7f13984e7120]
#7: /mnt/jdk11-openj9-0.18.1/lib/compressedrefs/libj9jit29.so(+0x256f4f) [0x7f13984e7f4f]
...

Therefore I am afraid that this incident cannot be closed yet, as there is no released AdoptOpenJDK with OpenJ9 version that shows the performance improvement.

Please note this kind of crash is discussed in other incidents such as #8548

Kind regards, Alexandre Vermeerbergen

fjeremic commented 4 years ago

@avermeer this issue was fixed in #8598. Looks like the targeted release was 0.19.0 which is scheduled for release on March 17th, 2019. You can wait to try and pick up that release build, or attempt to measure with the latest nightly build from Adopt which will have the fix.

[1] https://projects.eclipse.org/projects/technology.openj9/releases/0.19.0

pshipton commented 4 years ago

Note that the 0.19.0 release is only for Java 14.