Native benchmark fails on 0.4.10

[CharlieTap]

I'm running:

Kotlin 1.9.2 OS: Sonoma CPU: Apple M1 Max Gradle: 8.5

You can recreate by running build on this branch:

https://github.com/CharlieTap/cachemap/tree/failing-benchmark

I get the following:

Execution failed for task ':benchmark:macosArm64BenchmarkGenerate'.
> There was a failure while executing work items
   > A failure occurred while executing kotlinx.benchmark.gradle.NativeSourceGeneratorWorker
      > 'void org.jetbrains.kotlin.serialization.deserialization.DeserializationComponents.<init>(org.jetbrains.kotlin.storage.StorageManager, org.jetbrains.kotlin.descriptors.ModuleDescriptor, org.jetbrains.kotlin.serialization.deserialization.DeserializationConfiguration, org.jetbrains.kotlin.serialization.deserialization.ClassDataFinder, org.jetbrains.kotlin.serialization.deserialization.AnnotationAndConstantLoader, org.jetbrains.kotlin.descriptors.PackageFragmentProvider, org.jetbrains.kotlin.serialization.deserialization.LocalClassifierTypeSettings, org.jetbrains.kotlin.serialization.deserialization.ErrorReporter, org.jetbrains.kotlin.incremental.components.LookupTracker, org.jetbrains.kotlin.serialization.deserialization.FlexibleTypeDeserializer, java.lang.Iterable, org.jetbrains.kotlin.descriptors.NotFoundClasses, org.jetbrains.kotlin.serialization.deserialization.ContractDeserializer, org.jetbrains.kotlin.descriptors.deserialization.AdditionalClassPartsProvider, org.jetbrains.kotlin.descriptors.deserialization.PlatformDependentDeclarationFilter, org.jetbrains.kotlin.protobuf.ExtensionRegistryLite, org.jetbrains.kotlin.types.checker.NewKotlinTypeChecker, org.jetbrains.kotlin.resolve.sam.SamConversionResolver, java.util.List, org.jetbrains.kotlin.serialization.deserialization.EnumEntriesDeserializationSupport, int, kotlin.jvm.internal.DefaultConstructorMarker)'

[fzhinkin]

@CharlieTap I tried to reproduce the problem and the build failed with two different issues none of which match the summary you posted above.

Could you please share your build log via --scan?

The two aforementioned issues I faced while running the benchmarks are:

• for me, benchmark tasks are running with kotlin-compiler-embeddable:1.8.20 instead of 1.9.20 and it just can't resolve the klib:

• What went wrong: Execution failed for task ':benchmark:macosArm64BenchmarkGenerate'.

  There was a failure while executing work items A failure occurred while executing kotlinx.benchmark.gradle.NativeSourceGeneratorWorker e: Could not find "/Users/Filipp.Zhinkin/Development/cachemap/benchmark/build/classes/kotlin/macosArm64/main/klib/benchmark.klib" in [/Users/Filipp.Zhinkin/Development/cachemap]

• once I overcome the issue with the wrong compiler version on the class path, benchmarks generation failed because multiple benchmark modes are implicitly unsupported for native targets (https://github.com/Kotlin/kotlinx-benchmark/blob/3f32f2d9d9732a2a01d231134c52e6554428b519/plugin/main/src/kotlinx/benchmark/gradle/SuiteSourceGenerator.kt#L142C26-L142C26).

[CharlieTap]

I've posted a fresh branch here with it failing here, with the extra mode the removed (this should probably be documented as the annotations are copied from JMH so I won't be the last to run into this)

https://github.com/CharlieTap/cachemap/tree/failing-native-benchmark

And also heres a scan:

https://scans.gradle.com/s/lazrbivpp5g7s

With regards to kotlin compiler embeddable I'm not sure what you're referring to as 1.8.20 is not a dependency of my project at all?

[fzhinkin]

@CharlieTap thanks for posting the scan!

With regards to kotlin compiler embeddable I'm not sure what you're referring to as 1.8.20 is not a dependency of my project at all?

It's a transitive dependency for coroutines. In your case it seems to be the similar issue, but with serialization. I'm not quite sure why it's happening (Gradle reports that 1.9. versions of these artifacts should be used instead of 1.8.), I'll check what could be done there.

[fzhinkin]

Filed https://github.com/Kotlin/kotlinx-benchmark/issues/174 for the issue with multiple benchmark modes.

[fzhinkin]

It's a transitive dependency for coroutines

Looking at the benchmark module, it's from the kotlinter:

$ ./gradlew :benchmark:buildEnvironment
...
+--- org.jmailen.kotlinter:org.jmailen.kotlinter.gradle.plugin:3.15.0
|    \--- org.jmailen.gradle:kotlinter-gradle:3.15.0
|         +--- com.pinterest.ktlint:ktlint-core:0.49.1
|         |    +--- org.jetbrains.kotlin:kotlin-compiler-embeddable:1.8.20
...
|         +--- com.pinterest.ktlint:ktlint-rule-engine:0.49.1
|         |    +--- com.pinterest.ktlint:ktlint-logger:0.49.1
...
|         |    +--- org.jetbrains.kotlin:kotlin-compiler-embeddable:1.8.20 (*)
...

[fzhinkin]

@CharlieTap could you please try placing the following block before plugins in benchmark's build.gradle.kts?

buildscript {
    configurations.all {
        resolutionStrategy {
            eachDependency {
                if (requested.group == "org.jetbrains.kotlin") {
                    val kv = libs.versions.kotlin.get()
                    logger.warn("${requested.group}:${requested.name}:${requested.version} --> $kv")
                    useVersion(kv)
                }
            }
        }
    }
}

[CharlieTap]

Ohh thats fixes it!

So both ktlint and this library use the embedded compiler and its picking the 1st one loaded on the classpath? Is there a fix for this other than forcing the version like you provided above?

[fzhinkin]

The problem is that benchmarks plugin declares embedded compiler as a compile-only dependency (and there's a good reason for doing that) and that may need to use of incompatible compiler version in runtime. We'll check what could be done here to prevent the problem in general.

[qurbonzoda]

There is an ongoing effort from the Build Tools team to resolve this issue: https://youtrack.jetbrains.com/issue/KT-66764