This PR adds a "canary" build to the performance regression CI of OpenJDK. The "canary" is a chosen revision of mmtk-core and mmtk-openjdk that is tested alongside each merged PR. The performance of the "canary" should not change unless there is an environment change or there is a noise. Spotting a change in the "canary" performance can help us identify environment changes that are unintended or otherwise unnoticed, and also identify the noise level.
Currently, we choose a specific release version as the version of the "canary". Using a release version has the advantage of being easy to specify the exact revision of both the mmtk-core and the mmtk-openjdk repository. We may also switch to some methods of automatically select the canary version in the future.
There are other minor changes made.
We slightly change the directory structure. We create two directory, namely latest and canary. In each of the directories, we check out mmtk-core and mmtk-openjdk of the latest and the canary versions, respectively.
We use the ci-replace-mmtk-dep.py script to replace the revision of the mmtk-core dependency in mmtk-openjdk. As a result, we no longer need to use sed, and no longer need to copy the mmtk-core directory into mmtk-openjdk/repos.
We no longer set the RUSTUP_TOOLCHAIN environment variable because
The latest and the canary version may not use the same toolchain, and,
the right toolchain will be chosen when running the cargo command according to the rust-toolchain file in the directory.
This PR adds a "canary" build to the performance regression CI of OpenJDK. The "canary" is a chosen revision of mmtk-core and mmtk-openjdk that is tested alongside each merged PR. The performance of the "canary" should not change unless there is an environment change or there is a noise. Spotting a change in the "canary" performance can help us identify environment changes that are unintended or otherwise unnoticed, and also identify the noise level.
Currently, we choose a specific release version as the version of the "canary". Using a release version has the advantage of being easy to specify the exact revision of both the mmtk-core and the mmtk-openjdk repository. We may also switch to some methods of automatically select the canary version in the future.
There are other minor changes made.
latest
andcanary
. In each of the directories, we check outmmtk-core
andmmtk-openjdk
of the latest and the canary versions, respectively.ci-replace-mmtk-dep.py
script to replace the revision of themmtk-core
dependency inmmtk-openjdk
. As a result, we no longer need to usesed
, and no longer need to copy themmtk-core
directory intommtk-openjdk/repos
.RUSTUP_TOOLCHAIN
environment variable becausecargo
command according to therust-toolchain
file in the directory.