adoptium / aqa-tests

Home of test infrastructure for Adoptium builds
https://adoptium.net/aqavit
Apache License 2.0
129 stars 308 forks source link

Enable perf testing against jlink NRE/bundle with JRE as baseline #1933

Open smlambert opened 4 years ago

smlambert commented 4 years ago

Related: https://github.com/AdoptOpenJDK/openjdk-build/pull/1812

As we are about to build an 'adopt_bundle' we should verify that it is performant (startup and runtime metrics). We can also measure JRE perf and compare against it using various performance benchmarks we have in the suite.

Since the project now has some bare-metal machines for some platforms (labelled ci.role.perf), we will focus our runs on those machines/platforms.

Initially we can run these tests in Grinders, but need to incorporate them into our overall pipeline story and configure the TRSS server (trss.adoptopenjdk.net) to monitor the runs so we can store historical data and watch for trends/regressions.

aahlenst commented 4 years ago

Artifacts to test for 11/x64:

Artifacts to test for 15/x64:

smlambert commented 3 years ago

Perf runs on ci.role.perf&&hw.arch.x86&&sw.os.linux machines, using newly created Grinder_Perf job in order to differentiate from heavily used Grinder for monitoring via TRSS Note: dacapo is shortened to dac and renaissance to ren.


sanity.perf Initial Results for 11/x64

Object Under Test dac-h2 dac-lusearch-fix ren-akka-uct ren-fj-kmeans ren-future-genetic ren-naive-bayes ren-scala-kmeans
JDK / Rerun
JRE
jlink compress=0
jlink compress=1
jlink compress=2

sanity.perf Initial Results for 15/x64

Object Under Test dac-h2 dac-lusearch-fix ren-akka-uct ren-fj-kmeans ren-future-genetic ren-naive-bayes ren-scala-kmeans
JDK
JRE
jlink compress=0
jlink compress=1
jlink compress=2
smlambert commented 3 years ago

I will plan to break this up into sections and divide it amongst GHC open-source day participants. (fyi @llxia @sophia-guo)

sophia-guo commented 3 years ago

Note: jlink links in https://github.com/adoptium/aqa-tests/issues/1933#issuecomment-671942884 doesn't work anymore?

aahlenst commented 3 years ago

@sophia-guo Yes, I canceled the account hosting the binaries. They would not have been of much use anyway because of their age. Recreating them shouldn't be very difficult: Dump the list of modules with java --list-modules on the legacy JRE image and throw away what looks fishy on JRE 14 and newer. Back then, some parts started to leak into the JRE from the JDK. I'm not sure if that's still the case.