Kotlin / kotlinx-benchmark

Kotlin multiplatform benchmarking toolkit
Apache License 2.0
499 stars 40 forks source link

Support setting up different toolchains for different benchmark configurations #187

Open fzhinkin opened 7 months ago

fzhinkin commented 7 months ago

(Spin-off from the #176)

When working on performance improvements, sometimes there's a need to validate performance with different runtime versions. It would be nice to support per-configuration runtime version selection. Something like:

configurations {
    create("jdk17") {
        mode = "avgt"
        outputTimeUnit = "ns"
        include("MyPreciousBenchmark")
        jvmToolchain {
            languageVersion.set(JavaLanguageVersion.of(17))
        }
    }
    create("jdk21") {
        mode = "avgt"
        outputTimeUnit = "ns"
        include("MyPreciousBenchmark")
        jvmToolchain {
            languageVersion.set(JavaLanguageVersion.of(21))
        }
    }
    create("nodeJs21") {
        mode = "avgt"
        outputTimeUnit = "ns"
        include("MyPreciousBenchmark")
        nodeJsVersion = "21.0.0"
    }
}

However, there are a few obvious issues:

Not suggesting a particular solution, filled the issue to ensure the idea won't be lost.