Support Kotlin 2.0/Multiplatform artifacts during dependency resolution

[lihaoyi]

Follow up to https://github.com/com-lihaoyi/mill/issues/3611

The basic issue is that Coursier is unable to resolve Kotlin 2.0 dependencies, such as https://repo1.maven.org/maven2/org/jetbrains/kotlinx/kotlinx-html-js/0.11.0/. These dependencies do not contain .jar files, but instead contain .module files and .klib files:

kotlin-stdlib-js-2.0.0-sources.jar                2024-05-20 16:12    620420      
kotlin-stdlib-js-2.0.0-sources.jar.asc            2024-05-20 16:12       679      
kotlin-stdlib-js-2.0.0-sources.jar.md5            2024-05-20 16:12        32      
kotlin-stdlib-js-2.0.0-sources.jar.sha1           2024-05-20 16:12        40      
kotlin-stdlib-js-2.0.0.klib                       2024-05-20 16:12   2635364      
kotlin-stdlib-js-2.0.0.klib.asc                   2024-05-20 16:12       679      
kotlin-stdlib-js-2.0.0.klib.md5                   2024-05-20 16:12        32      
kotlin-stdlib-js-2.0.0.klib.sha1                  2024-05-20 16:12        40      
kotlin-stdlib-js-2.0.0.module                     2024-05-20 16:12      2987      
kotlin-stdlib-js-2.0.0.module.asc                 2024-05-20 16:12       679      
kotlin-stdlib-js-2.0.0.module.md5                 2024-05-20 16:12        32      
kotlin-stdlib-js-2.0.0.module.sha1                2024-05-20 16:12        40      
kotlin-stdlib-js-2.0.0.pom                        2024-05-20 16:12      1544      
kotlin-stdlib-js-2.0.0.pom.asc                    2024-05-20 16:12       679      
kotlin-stdlib-js-2.0.0.pom.md5                    2024-05-20 16:12        32      
kotlin-stdlib-js-2.0.0.pom.sha1                   2024-05-20 16:12        40      
kotlin-stdlib-js-2.0.0.spdx.json                  2024-05-20 16:12      1064      
kotlin-stdlib-js-2.0.0.spdx.json.asc              2024-05-20 16:12       679      
kotlin-stdlib-js-2.0.0.spdx.json.md5              2024-05-20 16:12        32      
kotlin-stdlib-js-2.0.0.spdx.json.sha1             2024-05-20 16:12        40      

The success criteria is that after applying the following diff

lihaoyi mill$ git diff
diff --git a/example/kotlinlib/web/3-hello-kotlinjs/build.mill b/example/kotlinlib/web/3-hello-kotlinjs/build.mill
index 950fec1eb2..a85a0396a2 100644
--- a/example/kotlinlib/web/3-hello-kotlinjs/build.mill
+++ b/example/kotlinlib/web/3-hello-kotlinjs/build.mill
@@ -14,6 +14,9 @@ object foo extends KotlinJSModule {
   def moduleKind = ModuleKind.ESModule
   def kotlinVersion = "1.9.25"
   def kotlinJSRunTarget = Some(RunTarget.Node)
+  def ivyDeps = Agg(
+    ivy"org.jetbrains.kotlinx:kotlinx-html-js:0.11.0",
+  )
   object test extends KotlinJSModule with KotlinJSKotlinXTests
 }

diff --git a/example/kotlinlib/web/3-hello-kotlinjs/foo/src/foo/Hello.kt b/example/kotlinlib/web/3-hello-kotlinjs/foo/src/foo/Hello.kt
index 09f3ccd16a..7fdfa75499 100644
--- a/example/kotlinlib/web/3-hello-kotlinjs/foo/src/foo/Hello.kt
+++ b/example/kotlinlib/web/3-hello-kotlinjs/foo/src/foo/Hello.kt
@@ -1,4 +1,12 @@
 package foo
+import kotlinx.browser.document
+import kotlinx.browser.window
+import kotlinx.html.a
+import kotlinx.html.div
+import kotlinx.html.dom.append
+import kotlinx.html.dom.create
+import kotlinx.html.p
+

 fun getString() = "Hello, world"

The following command should not produce a compiler error:

lihaoyi mill$ ./mill -i dist.run example/kotlinlib/web/3-hello-kotlinjs foo.compile
[2473/2473] dist.run
[31/31] foo.compile
[31] Compiling 1 Kotlin sources to /Users/lihaoyi/Github/mill/example/kotlinlib/web/3-hello-kotlinjs/out/foo/compile.dest/classes ...
[31] /Users/lihaoyi/Github/mill/example/kotlinlib/web/3-hello-kotlinjs/foo/src/foo/Hello.kt:2:8: error: unresolved reference: kotlinx
[31] import kotlinx.browser.document
[31]        ^
[31] /Users/lihaoyi/Github/mill/example/kotlinlib/web/3-hello-kotlinjs/foo/src/foo/Hello.kt:3:8: error: unresolved reference: kotlinx
[31] import kotlinx.browser.window
[31]        ^
[31] /Users/lihaoyi/Github/mill/example/kotlinlib/web/3-hello-kotlinjs/foo/src/foo/Hello.kt:4:8: error: unresolved reference: kotlinx
[31] import kotlinx.html.a
[31]        ^
[31] /Users/lihaoyi/Github/mill/example/kotlinlib/web/3-hello-kotlinjs/foo/src/foo/Hello.kt:5:8: error: unresolved reference: kotlinx
[31] import kotlinx.html.div
[31]        ^
[31] /Users/lihaoyi/Github/mill/example/kotlinlib/web/3-hello-kotlinjs/foo/src/foo/Hello.kt:6:8: error: unresolved reference: kotlinx
[31] import kotlinx.html.dom.append
[31]        ^
[31] /Users/lihaoyi/Github/mill/example/kotlinlib/web/3-hello-kotlinjs/foo/src/foo/Hello.kt:7:8: error: unresolved reference: kotlinx
[31] import kotlinx.html.dom.create
[31]        ^
[31] /Users/lihaoyi/Github/mill/example/kotlinlib/web/3-hello-kotlinjs/foo/src/foo/Hello.kt:8:8: error: unresolved reference: kotlinx
[31] import kotlinx.html.p
[31]        ^
[31/31] ================================================ foo.compile ================================================= 3s
1 tasks failed
foo.compile Kotlin compiler failed with exit code 1 (COMPILATION_ERROR)
[2473/2473] ==================== dist.run example/kotlinlib/web/3-hello-kotlinjs foo.compile ========================= 6s
1 tasks failed
dist.run dev.run failed with an exception. Interactive Subprocess Failed (exit code 1)

[lihaoyi]

CC @alexarchambault. Do you think this something we could configure Coursier to do without patching it? Or would this functionality need to be added to Coursier itself?

[alexarchambault]

Coursier should support that, it's a matter of asking for the right types and extensions. I can have a look a bit later today.

[0xnm]

Probably it makes sense to add a bit more info on how Kotlin Multiplatform ecosystem works, it may be useful in the future for the artifact filtering. In fact, Kotlin Multiplatform heavily relies on the Gradle Module metadata (this is what gets published as .module file) to know what to fetch.

When Kotlin Multiplatform publishes artifacts for multiple targets, it has target-specific modules + root module (described here).

This can be seen on the example of kotlinx-html library:

The root artifact has the necessary metadata to point to the platform-specific artifacts (like kotlinx-html-js, kotlinx-html-iossimulatorarm64, etc.), that allows to specify only root artifact instead of pointing to the each target-specific artifact individually. In Gradle terms I can do something like:

commonMain.dependencies {
  implementation("org.jetbrains.kotlin:kotlinx-html:x.x.x"
}

or even (if we are not building common code):

jsMain.dependencies {
  implementation("org.jetbrains.kotlin:kotlinx-html:x.x.x"
}

and then if I have js, iosArm64, etc. targets registered the proper artifacts will be pulled. Instead of doing:

jsMain.dependencies {
  implementation("org.jetbrains.kotlin:kotlinx-html-js:x.x.x"
}

And then target-specific artifact .module file contains exact location and alias for the library file.

[lihaoyi]

Thanks for the details @0xnm! We can definitely look at whether we can support the .module files directly in Mill or if @alexarchambault can put some support upstream into Coursier