Fix kotlin gradle plugin incompatibilities with versions

[NoPhaseNoKill]

Original build scan for failures: https://scans.gradle.com/s/husk3ljmjerjs/console-log?anchor=78&page=1

1. Fixes a whole bunch of warnings which were going unnoticed
2. Ensures the kotlin-gradle-plugin uses the embedded version so that we don't get into race conditions which cause the warning about embedded kotlin plugin version relying on 1.9.24 but our build using 2.0.20 which may be problematic

My other thoughts/questions:

Why are versions and aggregations subprojects and NOT included builds?

The current problem is that by not exposing them as an includedBuild, to my knowledge, you can't control (unless through locking mechanisms), when gradle may reach into a subproject. Because ALL subprojects of this particular build are dependent on the explicit constraints of the kotlin gradle plugin, when you don't provide means of locking the versions (by an explicit dependency through an includedBuild or otherwise) - you run into the exact race condition I saw. However, if you either:

• Had an included build for the versions which was ordered first, where the plugins relied on that dependency's output, you could have a task which created the embedded kotlin JAR's, and used these for the remaining plugins
• Added a configuration from a subproject, which you then required/needed for the plugins project (essentially the same thing as an included build), you could have a producer -> consumer relationship which would enable you to contro the versions.

One of the continual issues I've had with gradle, and one of the MAJOR problems with it IMO, is the way that the kotlin gradle plugin manages it's versioning. Their BOM is completely awful, the plugin offers no means out of the box for protecting you against exactly what you've done, and it's bitten me several times now. I came to this project to find out how you'd solved it, and funnily enough realised that it's probably far more common than people think/realise if even you're getting it wrong (one of the gradle experts).

What are your thoughts on something like the following:

// consumer build file

plugins {
    `java-gradle-plugin`
    `maven-publish`
    `kotlin-dsl`
}

version = "1.0.0-local"
group = "com.nophasenokill"

gradlePlugin {
    plugins {
        create("consumerPlugin") {
            id = "com.nophasenokill.consumer"
            implementationClass = "com.nophasenokill.ConsumerPlugin"
        }
    }
}

repositories {
    mavenCentral()
}

dependencies {
    implementation(project(":producer", configuration = "dependencyImplementationConfiguration"))

    testImplementation(project(":producer"))
    testRuntimeOnly(project(":producer"))
}

tasks.register("assertClasspaths") {
    val compileClasspath: FileCollection = configurations.compileClasspath.get()
    val runtimeClasspath: FileCollection = configurations.runtimeClasspath.get()
    val testCompileClasspath: FileCollection = configurations.testCompileClasspath.get()
    val testRuntimeClasspath: FileCollection = configurations.testRuntimeClasspath.get()
    inputs.files(compileClasspath, runtimeClasspath, testCompileClasspath, testRuntimeClasspath )

    val expectedKotlinJars = listOf(
        "kotlin-stdlib-1.9.24.jar",
        "kotlin-reflect-1.9.24.jar",
    )

    doLast {
        expectedKotlinJars.forEach { expectedJar ->
            val hasExpectedKotlinInCompile = compileClasspath.files.any { it.name === expectedJar }
            val hasExpectedKotlinInTestCompile = testCompileClasspath.files.any { it.name === expectedJar}
            val hasExpectedKotlinInRuntime = runtimeClasspath.files.any { it.name === expectedJar }
            val hasExpectedKotlinInTestRuntime = testRuntimeClasspath.files.any { it.name === expectedJar }

            assert(hasExpectedKotlinInCompile)
            assert(hasExpectedKotlinInTestCompile)
            assert(hasExpectedKotlinInRuntime)
            assert(hasExpectedKotlinInTestRuntime)
        }
    }
}

// producer build file

plugins {
    `java-gradle-plugin`
    `maven-publish`
    `kotlin-dsl`
}

gradlePlugin {
    plugins {
        create("producerPlugin") {
            id = "com.nophasenokill.producer"
            implementationClass = "com.nophasenokill.ProducerPlugin"
        }
    }
}

version = "1.0.0-local"
group = "com.nophasenokill"

repositories {
    mavenCentral()
}

val dependencyImplementationConfiguration: Configuration by configurations.creating {
    extendsFrom(configurations.implementation.get())
    isCanBeResolved = false
    isCanBeConsumed = true
}

dependencies {
    implementation(platform("org.jetbrains.kotlin:kotlin-bom:${embeddedKotlinVersion}"))
    dependencyImplementationConfiguration("org.jetbrains.kotlin:kotlin-gradle-plugin")

    testImplementation(platform("org.junit:junit-bom:5.10.1"))
    testImplementation("org.junit.jupiter:junit-jupiter")
    testRuntimeOnly("org.junit.platform:junit-platform-launcher")
}

tasks.test {
    useJUnitPlatform()
}

When running assertClasspaths you effectively get safety that things are working correctly. Having this be a 'meta-task' would then possibly allow you to get into the lifecycle early, preventing these frustrating gradle kotlin version errors

[jjohannes]

Thank you for providing this PR. The example code was just added to have some code in the project to run the build. I did not pay much attention to it. I appreciate it being cleaned up.

I didn't get around to do updates here in the past weeks. I updated things now. For that, I always force push the branches that are not main, as all of these are variations of main.

There are conflicts now, due to the force push. If you have a minute, maybe you can reset the branch to the latest.

I am about to leave for holidays so just one point regarding the issues/questions:

• There should be no difference for the dependency management if something is a subproject or an includedBuild. If it is, it's coincidence due to some ordering. But order should not matter in a consistent setup. There must be something else going on.

When I am back, I'll have a closer look.

[NoPhaseNoKill]

Your changes effectively invalidate mine, because of the gradle version bump.

The actual root cause/original issue is:

1. When you want to control the version of the kotlin JVM plugin inside of a convention plugin, you need a way of upgrading the kotlin-dsl version to match the same version.
2. This is not always possible, due to the gradle version enforcing a particular kotlin version for the transitive dependency of kotlin-dsl. For instance, with the old version of gradle you had (8.10.0) - the dsl version was: org.gradle.kotlin.kotlin-dsl:org.gradle.kotlin.kotlin-dsl.gradle.plugin:4.5.0. I wanted a version of kotlin("jvm") 2.0.20 -> which is only compatible with dsl 5.1.1 -> but is overriden due to the kotlin-dsl plugin loaded first.

Some of the issues this can cause is:

• Incorrect versions of your expected kotlin version, for the jvm, but ONLY if you have convention plugins (as the kotlin-dsl is needed at a stage that is too early to know about any other versions, because it's a direct dependency of the builds scripts)
• Strange warnings which don't make sense with what the user expects (1.9.24 features of the kotlin dsl are required for this build, but version 2.0.20 is being used etc)

An example is shown here: https://scans.gradle.com/s/augewr4vriccu/build-dependencies?expandAll&toggled=W1syLDAsWzY4XV1d

Where you can see the build dependencies are:

org.gradle.kotlin.kotlin-dsl:org.gradle.kotlin.kotlin-dsl.gradle.plugin:4.5.0 // incompatibile wqith 2.0.20 org.jetbrains.kotlin:kotlin-stdlib:{strictly 1.9.24} // incompatible with 2.0.20

:build-logic org.jetbrains.kotlin:kotlin-gradle-plugin:2.0.20 // This not compatible with kotlin-dsl 4.5.0

You can also see in the build scan a mismatch of versions of the configuration kotlinCompilerPluginClasspathMain

// build-logic
org.jetbrains.kotlin:kotlin-assignment-compiler-plugin-embeddable:1.9.24
org.jetbrains.kotlin:kotlin-sam-with-receiver-compiler-plugin-embeddable:1.9.24
org.jetbrains.kotlin:kotlin-scripting-compiler-embeddable:1.9.24

versus

//sub-project-one

org.jetbrains.kotlin:kotlin-scripting-compiler-embeddable:2.0.20
org.jetbrains.kotlin:kotlin-scripting-compiler-impl-embeddable:2.0.20

I've included the minimum reproducible example if what I'm saying doesn't make sense.

kotlin-dsl-version-conflict-demo.zip

There are ways to fix this, but I feel like the only 'nice' way is:

1. Include a task which prints the embedded version to a file
2. If the embedded version mismatches against any transitive dependency - possibly throw an error/fix it automartically depending on the control you want

This way we can just declare a single kotlin version, and if it can't do it - will throw an error, or will make it all work - rather than giving us inconsistent versions.

edit: Show further dependency discrepancy on build scan