google / ksp

Kotlin Symbol Processing API
https://github.com/google/ksp
Apache License 2.0
2.87k stars 269 forks source link

KSP error when using Gradle JVM Toolchain #740

Open alexspence opened 2 years ago

alexspence commented 2 years ago

Created a project to reproduce this: https://github.com/alexspence/repro-ksp-toolchain

Repro Steps:

Run the following command using JDK 1.8 then JDK 17

./gradlew --no-build-cache clean build

Expected Result:

Build output is the same due to toolchain using JDK 17

Actual Result:

JDK 1.8 Fails because KSP Doesn't seem to pickup the Toolchain configuration

My Local Output:

❯ jenv shell 17
❯ ./gradlew --no-build-cache clean build

BUILD SUCCESSFUL in 3s
18 actionable tasks: 18 executed
❯ jenv shell 1.8
❯ ./gradlew --no-build-cache clean build

> Task :consumer:kspKotlin FAILED
e: java.lang.UnsupportedClassVersionError: example/annoation/processor/ExampleProcessorProvider has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 52.0
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:757)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.jetbrains.kotlin.cli.jvm.plugins.ServiceLoaderLite.loadImplementations(ServiceLoaderLite.kt:51)
        at org.jetbrains.kotlin.cli.jvm.plugins.ServiceLoaderLite.loadImplementations(ServiceLoaderLite.kt:44)
        at com.google.devtools.ksp.KotlinSymbolProcessingExtension.loadProviders(KotlinSymbolProcessingExtension.kt:68)
        at com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:161)
        at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:120)
        at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:96)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:262)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:53)
        at org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:113)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:253)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:100)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli$default(KotlinToJVMBytecodeCompiler.kt:58)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:170)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:52)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:92)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:44)
        at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:98)
        at org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1618)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
        at sun.rmi.transport.Transport$1.run(Transport.java:200)
        at sun.rmi.transport.Transport$1.run(Transport.java:197)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':consumer:kspKotlin'.
> Internal compiler error. See log for more details

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s
15 actionable tasks: 15 executed
samzurcher commented 2 years ago

We are running into the same problem. Is there any chance you guys could look into this and support toolchains?

pablobaldez commented 1 year ago

I'm also with this issue. Which info do I need to send to help you to understand why?

Eklow02 commented 1 year ago

Same here. Also running into this issue

sebek64 commented 9 months ago

I also hit this issue. Ksp plugin simply uses java from PATH, not the one from JAVA_HOME environment variable. So even without gradle jvm toolchain it is broken.

austinarbor commented 5 months ago

I can't figure this one out for the life of me - I have Java 21 in my PATH, JAVA_HOME , and my toolchain, but the only way I can get it to work is if I specifically set the java version of the processor project to Java 17.

has anyone figured this one out?

howard-paypay commented 2 months ago

I encountered this issue today. It seems like the cause is ksp has its own daemon and it gets picked up by gradle despite running a different jdk version.

By killing the KotlinCompileDaemon process, it works again.

ianbrandt commented 2 months ago

Same here in local and CI environments.

I'm not using Java toolchains. I'm launching the Gradle Wrapper with JAVA_HOME set for a given build plan, and relying on --release to set the desired Java compilation version.

If a KotlinCompileDaemon is running leftover from a JAVA_HOME= Java 11 build, I get this failure when a JAVA_HOME= Java 17 build runs.

It's particularly problematic in CI, as I don't have control over when build plans configured for different JDKs launch, nor ready access to log in and kill a running KotlinCompileDaemon. I suppose one workaround might be to run plans for different JDKs under different user accounts on the build agents, but that's not our current setup.

Is this actually a KSP issue, or is it a Kotlin Gradle Plugin/Compiler issue? It seems to me that work shouldn't be delegated to a KotlinCompileDaemon that is executing with a different JVM (or at least one with a lower major version) than the current Gradle build JVM.

e: java.lang.UnsupportedClassVersionError: my/package/MyProcessorProvider has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0
        at java.base/java.lang.ClassLoader.defineClass1(Native Method)
        at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022)
        at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
        at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:555)
        at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:458)
        at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:452)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:451)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:398)
        at org.jetbrains.kotlin.util.ServiceLoaderLite.loadImplementations(ServiceLoaderLite.kt:51)
        at org.jetbrains.kotlin.util.ServiceLoaderLite.loadImplementations(ServiceLoaderLite.kt:44)
        at com.google.devtools.ksp.KotlinSymbolProcessingExtension.loadProviders(KotlinSymbolProcessingExtension.kt:89)
        at com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:265)
        at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:112)
        at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:77)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:256)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler$analyze$1.invoke(KotlinToJVMBytecodeCompiler.kt:247)
        at org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:115)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:247)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:87)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli$default(KotlinToJVMBytecodeCompiler.kt:43)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:165)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:50)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:104)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:48)
        at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:101)
        at org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1523)

Windows 10.0.19045. Gradle 8.10. Kotlin 1.9.23. KSP 1.9.23-1.0.19.

ianbrandt commented 2 months ago

Pasting some additional details from Slack for posterity:

My immediate scenario is that I'm working on upgrading my project from Java 11 to Java 17. I created a branch and made the requisite code changes. When I pushed that branch and setup a CI plan to build it with Java 17, the build failed on the same error from this bug report. My assumption is that a KotlinCompileDaemon invoked with Java 11 was already running idle on the given build agent from a recent main branch build.

Another scenario where I could potentially foresee this occurring is if I have a maintenance branch for my project that's on an older Java version than the main branch. Someone pushes a bug fix to the maintenance branch. There's no idle KotlinCompileDaemon at the time, so one is launched with Java 11. A later push to the main branch triggers a Java 17 build, and it tries to use that idle Java 11 daemon.

I seem to remember also running into this issue in the past when I had the wrong Project SDK configured for a project in IntelliJ, and then corrected it and ran another build.

I've reproduced this with both Kotlin/KSP 1.9.23-1.0.19 and 2.0.10-1.0.24 via a couple of branches of a toy project.

Java 17:

Java 21:

In a Java 17 terminal session:

% git switch java-17-K2
Switched to branch 'java-17-K2'
% java --version       
openjdk 17.0.12 2024-07-16
OpenJDK Runtime Environment Temurin-17.0.12+7 (build 17.0.12+7)
OpenJDK 64-Bit Server VM Temurin-17.0.12+7 (build 17.0.12+7, mixed mode)
% ./gradlew --stop     
No Gradle daemons are running.
% ./gradlew --no-build-cache clean build
Starting a Gradle Daemon, 6 stopped Daemons could not be reused, use --status for details
Configuration on demand is an incubating feature.
Calculating task graph as configuration cache cannot be reused because JVM has changed.
Type-safe project accessors is an incubating feature.
[...]
BUILD SUCCESSFUL in 25s
69 actionable tasks: 65 executed, 4 up-to-date
Configuration cache entry stored.

Then in a Java 21 terminal session:

% git switch java-21-K2                 
Switched to branch 'java-21-K2'
Your branch is up to date with 'origin/java-21-K2'.
% java --version       
openjdk 21.0.4 2024-07-16 LTS
OpenJDK Runtime Environment Temurin-21.0.4+7 (build 21.0.4+7-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.4+7 (build 21.0.4+7-LTS, mixed mode)
% ./gradlew --no-build-cache clean build
Starting a Gradle Daemon, 1 incompatible and 6 stopped Daemons could not be reused, use --status for details
Configuration on demand is an incubating feature.
Calculating task graph as configuration cache cannot be reused because JVM has changed.
Type-safe project accessors is an incubating feature.

> Task :ksp-builder-generator:processor:test-project:kspKotlin FAILED
e: java.lang.UnsupportedClassVersionError: org/sdkotlin/buildergen/processor/BuilderGenSymbolProcessorProvider has been compiled by a more recent version of the Java Runtime (class file version 65.0), this version of the Java Runtime only recognizes class file versions up to 61.0
        at java.base/java.lang.ClassLoader.defineClass1(Native Method)
        at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1017)
        at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
        at java.base/java.net.URLClassLoader.defineClass(URLClassLoader.java:524)
        at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:427)
        at java.base/java.net.URLClassLoader$1.run(URLClassLoader.java:421)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:420)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:592)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:467)
        at org.jetbrains.kotlin.util.ServiceLoaderLite.loadImplementations(ServiceLoaderLite.kt:51)
        at org.jetbrains.kotlin.util.ServiceLoaderLite.loadImplementations(ServiceLoaderLite.kt:44)
        at com.google.devtools.ksp.KotlinSymbolProcessingExtension.loadProviders(KotlinSymbolProcessingExtension.kt:89)
        at com.google.devtools.ksp.AbstractKotlinSymbolProcessingExtension.doAnalysis(KotlinSymbolProcessingExtension.kt:265)
        at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration(TopDownAnalyzerFacadeForJVM.kt:112)
        at org.jetbrains.kotlin.cli.jvm.compiler.TopDownAnalyzerFacadeForJVM.analyzeFilesWithJavaIntegration$default(TopDownAnalyzerFacadeForJVM.kt:75)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze$lambda$12(KotlinToJVMBytecodeCompiler.kt:373)
        at org.jetbrains.kotlin.cli.common.messages.AnalyzerWithCompilerReport.analyzeAndReport(AnalyzerWithCompilerReport.kt:115)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.analyze(KotlinToJVMBytecodeCompiler.kt:364)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.runFrontendAndGenerateIrUsingClassicFrontend(KotlinToJVMBytecodeCompiler.kt:195)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:106)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:176)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:50)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:104)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:48)
        at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:101)
        at org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1556)
        at jdk.internal.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:360)
        at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200)
        at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
        at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:587)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:828)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:705)
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:704)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)

[...]

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':ksp-builder-generator:processor:test-project:kspKotlin'.
> A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
   > Internal compiler error. See log for more details

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Get more help at https://help.gradle.org.

BUILD FAILED in 16s
57 actionable tasks: 54 executed, 3 up-to-date
Configuration cache entry stored.

Watching in VisualVM, after the Java 17 session I have one new GradleDaemon process and two new KotlinCompileDaemon processes running, all Java 17. Then, after the Java 21 session, I have one new GradleDaemon on Java 21 (as Gradle identifies the existing Java 17 GradleDaemon is not compatible), but still just the same two KotlinCompileDaemon processes from before, both still Java 17 of course. There are no new Java 21 KotlinCompileDaemon processes.

Per, "The Kotlin daemon uses the same JDK that the Gradle daemon does", I would have expected new Java 21 KotlinCompileDaemon processes to have been launched along with the new Java 21 GradleDaemon.

With K2, KSP2 (ksp.useKSP2=true in 'gradle.properties'), given that it runs in the GradleDaemon instead of the KotlinCompileDaemon, seems to work around the issue in my reproducer project. I still don't see a new Java 21 KotlinCompileDaemon for a Java 21 build that follows a Java 17 build, so I do wonder if some other manifestation of the issue might still not crop up.

Unfortunately, my main project is currently blocked from upgrading to K2 by other open issues, so KSP2 (which itself is still just in beta), is not an option yet.

ianbrandt commented 2 months ago

Submitted upstream: https://youtrack.jetbrains.com/issue/KT-71048.

sschuberth commented 1 month ago

For the record, I can reproduce the issue also with ksp.useKSP2 = true. However, is this really an upstream issue? Isn't it instead a matter of KSP not adhering to the Java languageVersion as set via Gradle's toolchain mechanism?

sschuberth commented 1 month ago

Ok, I believe to have understood why ksp.useKSP2 = true alone is not enough: As explained here, "KSP2 runs in Gradle daemon", but the Gradle daemon is not affected by the Gradle toolchains mechanism. Instead, the daemon's JVM needs to be configured separately.

For me, the combination of

java {
    toolchain {
        languageVersion = JavaLanguageVersion.of(21)
    }
}

in the build.gradle.kts file, and

ksp.useKSP2 = true

in gradle.properties, and

toolchainVersion = 21

in gradle/gradle-daemon-jvm.properties did the trick to build successfully with Java 21.