sxa commented 3 years ago

Splitting this out from slack thread on the subject

I've been doing some analysis on the performance of the overnight builds and there are some things which are causing notable slowdowns which we should consider removing from the daily runs.

One of the biggest issues is with platforms which do not have a JIT. Now that we have enabled JIT on the AdoptOpenJDK arm32 JDK8 builds, the only ones build without an active JIT are OpenJ9 RISC-V (Still in development, not a full release platform yet) and JDK8 on Linux/s/390x for HotSpot. There are some others that are problematic too though - here is a list of ones we should consider modifying or removing:

JDK8/HS/s390x sanity.perf runs take around 9 hours, so I propose removing this, or trying to reduce the time of it via other means
JDK8/HS/xLinux sanity.external tomee test regularly hits 10 hour timeout (Although on the packet machine it can pass in 7 hours)
JDK11/J9/Linux-aarch64 extended.functional - takes 5 hours so is a lot longer than most other tests (although today it had another issue and hit the 10hr timeout)
JDK8/J9/s390x sanity.external - this has occasionally taken over 10 hours although today it seemed to get through most of the process then got stuck waiting for another executor near the end) - If this stays under about 90 minutes I'm ok to leave it in but we should keep an eye on it :-)

Linux/s390x JDK8 pipeline timings

I have run some new JDK8/s390x pipelines today and J9xl completed in about 2h31, j9 (non-XL) tool 4h30 but the difference to XL is likely just how it got scheduled around HotSpot as it has completed in about 2h31 in the past. I expect HotSpot will take about 10 hours including the build time and the sanity.perf run (Note that sanity.system takes over 4 hours for HotSpot so that will potentially be the limiting factor if we remove sanity.perf)

sxa commented 3 years ago

New related issue: systemtests on OpenJ9 have started running slowly over the weekend - raised at https://github.com/eclipse/openj9/issues/11080

sxa commented 3 years ago

Since we seem to be sprawling issues here:

Determing test schedule: https://github.com/AdoptOpenJDK/openjdk-tests/issues/2037
JDK8: https://github.com/AdoptOpenJDK/openjdk-tests/issues/2050
JDK11: https://github.com/AdoptOpenJDK/openjdk-tests/issues/2051

karianna commented 3 years ago

@smlambert for visibilty.

adoptium / ci-jenkins-pipelines

Remove sanity.perf from JDK8 pipelines (and other stuff...) #52

Linux/s390x JDK8 pipeline timings