search

FasterXML / jackson-module-kotlin

Module that adds support for serialization/deserialization of Kotlin (http://kotlinlang.org) classes and data classes.
Apache License 2.0
1.12k stars 175 forks source link

This module shouldn't bring kotlin-reflect 1.5 as a transitive dependency #566

Open Spikhalskiy opened 2 years ago

Spikhalskiy commented 2 years ago

Describe the bug Currently, when I build my project with Kotlin 1.4 and using this module, I get errors and warnings which are caused by this module bringing kotlin-reflect 1.5 as a transitive dependency into Kotlin 1.4 classpath and runtime:

> Task :temporal-kotlin:compileKotlin
w: Runtime JAR files in the classpath should have the same version. These files were found in the classpath:
    /Users/dmitry/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk8/1.4.32/3302f9ec8a5c1ed220781dbd37770072549bd333/kotlin-stdlib-jdk8-1.4.32.jar (version 1.4)
    /Users/dmitry/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk7/1.4.32/3546900a3ebff0c43f31190baf87a9220e37b7ea/kotlin-stdlib-jdk7-1.4.32.jar (version 1.4)
    /Users/dmitry/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-reflect/1.5.30/6773e974437dd6432aa646cc2f8ab71de42b5773/kotlin-reflect-1.5.30.jar (version 1.5)
    /Users/dmitry/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.5.30/d68efdea04955974ac1020f8f66ef8176bfbce1f/kotlin-stdlib-1.5.30.jar (version 1.5)
    /Users/dmitry/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-common/1.5.30/649ffab7767038323fec0cc41e2d7b0a8f65a378/kotlin-stdlib-common-1.5.30.jar (version 1.5)
w: Some runtime JAR files in the classpath have an incompatible version. Consider removing them from the classpath
e: org.jetbrains.kotlin.codegen.CompilationException: Back-end (JVM) Internal error: Couldn't inline method call 'toJavaDuration' into
@kotlin.time.ExperimentalTime public inline fun ConnectedWorkflowServiceStubs(timeout: kotlin.time.Duration, options: @io.temporal.kotlin.TemporalDsl() (io.temporal.serviceclient.WorkflowServiceStubsOptions.Builder.() -> kotlin.Unit)): io.temporal.serviceclient.WorkflowServiceStubs defined in io.temporal.serviceclient in file WorkflowServiceStubsExt.kt
/**
 * Create WorkflowService gRPC stubs using provided [options].
 *
 * @see WorkflowServiceStubs.newConnectedServiceStubs
 */
@ExperimentalTime
inline fun ConnectedWorkflowServiceStubs(
  timeout: Duration,
  options: @TemporalDsl WorkflowServiceStubsOptions.Builder.() -> Unit,
): WorkflowServiceStubs {
  return ConnectedWorkflowServiceStubs(timeout.toJavaDuration(), options)
}
Cause: Not generated
File being compiled: (99,48) in /Users/dmitry/Projects/temporal/java-sdk/temporal-kotlin/src/main/kotlin/io/temporal/serviceclient/WorkflowServiceStubsExt.kt
The root cause java.lang.IllegalStateException was thrown at: org.jetbrains.kotlin.codegen.inline.InlineCodegen$Companion.getCompiledMethodNodeInner(InlineCodegen.kt:596)
        at org.jetbrains.kotlin.codegen.inline.InlineCodegen.throwCompilationException(InlineCodegen.kt:105)
        at org.jetbrains.kotlin.codegen.inline.InlineCodegen.performInline(InlineCodegen.kt:149)
        at org.jetbrains.kotlin.codegen.inline.PsiInlineCodegen.genCallInner(PsiInlineCodegen.kt:67)
        at org.jetbrains.kotlin.codegen.CallGenerator$DefaultImpls.genCall(CallGenerator.kt:115)
        at org.jetbrains.kotlin.codegen.inline.PsiInlineCodegen.genCall(PsiInlineCodegen.kt:33)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.invokeMethodWithArguments(ExpressionCodegen.java:2772)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.invokeMethodWithArguments(ExpressionCodegen.java:2714)
        at org.jetbrains.kotlin.codegen.Callable$invokeMethodWithArguments$1.invoke(Callable.kt:42)
        at org.jetbrains.kotlin.codegen.Callable$invokeMethodWithArguments$1.invoke(Callable.kt:41)
        at org.jetbrains.kotlin.codegen.OperationStackValue.putSelector(StackValue.kt:79)
        at org.jetbrains.kotlin.codegen.StackValue.put(StackValue.java:125)
        at org.jetbrains.kotlin.codegen.StackValue.put(StackValue.java:118)
        at org.jetbrains.kotlin.codegen.inline.InlineCodegen.putArgumentOrCapturedToLocalVal(InlineCodegen.kt:375)
        at org.jetbrains.kotlin.codegen.inline.PsiInlineCodegen.putValueIfNeeded(PsiInlineCodegen.kt:187)
        at org.jetbrains.kotlin.codegen.inline.PsiInlineCodegen.genValueAndPut(PsiInlineCodegen.kt:158)
        at org.jetbrains.kotlin.codegen.CallBasedArgumentGenerator.generateExpression(CallBasedArgumentGenerator.kt:42)
        at org.jetbrains.kotlin.codegen.ArgumentGenerator.generate(ArgumentGenerator.kt:70)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.invokeMethodWithArguments(ExpressionCodegen.java:2743)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.invokeMethodWithArguments(ExpressionCodegen.java:2714)
        at org.jetbrains.kotlin.codegen.Callable$invokeMethodWithArguments$1.invoke(Callable.kt:42)
        at org.jetbrains.kotlin.codegen.Callable$invokeMethodWithArguments$1.invoke(Callable.kt:41)
        at org.jetbrains.kotlin.codegen.OperationStackValue.putSelector(StackValue.kt:79)
        at org.jetbrains.kotlin.codegen.StackValue.put(StackValue.java:125)
        at org.jetbrains.kotlin.codegen.StackValue.put(StackValue.java:118)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.putStackValue(ExpressionCodegen.java:442)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.lambda$visitReturnExpression$11(ExpressionCodegen.java:1752)
        at org.jetbrains.kotlin.codegen.OperationStackValue.putSelector(StackValue.kt:79)
        at org.jetbrains.kotlin.codegen.StackValueWithLeaveTask.putSelector(StackValue.kt:67)
        at org.jetbrains.kotlin.codegen.StackValue.put(StackValue.java:125)
        at org.jetbrains.kotlin.codegen.StackValue.put(StackValue.java:118)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.putStackValue(ExpressionCodegen.java:442)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.gen(ExpressionCodegen.java:423)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.returnExpression(ExpressionCodegen.java:1846)
        at org.jetbrains.kotlin.codegen.FunctionGenerationStrategy$FunctionDefault.doGenerateBody(FunctionGenerationStrategy.java:64)
        at org.jetbrains.kotlin.codegen.FunctionGenerationStrategy$CodegenBased.generateBody(FunctionGenerationStrategy.java:86)
        at org.jetbrains.kotlin.codegen.FunctionCodegen.generateMethodBody(FunctionCodegen.java:649)
        at org.jetbrains.kotlin.codegen.inline.PsiSourceCompilerForInline.generateMethodBody(SourceCompilerForInline.kt:206)
        at org.jetbrains.kotlin.codegen.inline.PsiSourceCompilerForInline.doCreateMethodNodeFromSource(SourceCompilerForInline.kt:304)
        at org.jetbrains.kotlin.codegen.inline.InlineCodegen$Companion.createInlineMethodNode$backend(InlineCodegen.kt:544)
        at org.jetbrains.kotlin.codegen.inline.InlineCodegen.performInline(InlineCodegen.kt:140)
        at org.jetbrains.kotlin.codegen.inline.PsiInlineCodegen.genCallInner(PsiInlineCodegen.kt:67)
        at org.jetbrains.kotlin.codegen.CallGenerator$DefaultImpls.genCall(CallGenerator.kt:115)
        at org.jetbrains.kotlin.codegen.inline.PsiInlineCodegen.genCall(PsiInlineCodegen.kt:33)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.invokeMethodWithArguments(ExpressionCodegen.java:2772)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.invokeMethodWithArguments(ExpressionCodegen.java:2714)
        at org.jetbrains.kotlin.codegen.Callable$invokeMethodWithArguments$1.invoke(Callable.kt:42)
        at org.jetbrains.kotlin.codegen.Callable$invokeMethodWithArguments$1.invoke(Callable.kt:41)
        at org.jetbrains.kotlin.codegen.OperationStackValue.putSelector(StackValue.kt:79)
        at org.jetbrains.kotlin.codegen.StackValue.put(StackValue.java:125)
        at org.jetbrains.kotlin.codegen.StackValue.put(StackValue.java:118)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.putStackValue(ExpressionCodegen.java:442)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.lambda$visitReturnExpression$11(ExpressionCodegen.java:1752)
        at org.jetbrains.kotlin.codegen.OperationStackValue.putSelector(StackValue.kt:79)
        at org.jetbrains.kotlin.codegen.StackValueWithLeaveTask.putSelector(StackValue.kt:67)
        at org.jetbrains.kotlin.codegen.StackValue.put(StackValue.java:125)
        at org.jetbrains.kotlin.codegen.StackValue.put(StackValue.java:118)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.putStackValue(ExpressionCodegen.java:442)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.gen(ExpressionCodegen.java:423)
        at org.jetbrains.kotlin.codegen.ExpressionCodegen.returnExpression(ExpressionCodegen.java:1846)
        at org.jetbrains.kotlin.codegen.FunctionGenerationStrategy$FunctionDefault.doGenerateBody(FunctionGenerationStrategy.java:64)
        at org.jetbrains.kotlin.codegen.FunctionGenerationStrategy$CodegenBased.generateBody(FunctionGenerationStrategy.java:86)
        at org.jetbrains.kotlin.codegen.FunctionCodegen.generateMethodBody(FunctionCodegen.java:649)
        at org.jetbrains.kotlin.codegen.FunctionCodegen.generateMethodBody(FunctionCodegen.java:484)
        at org.jetbrains.kotlin.codegen.FunctionCodegen.generateMethod(FunctionCodegen.java:260)
        at org.jetbrains.kotlin.codegen.FunctionCodegen.generateMethod(FunctionCodegen.java:165)
        at org.jetbrains.kotlin.codegen.FunctionCodegen.gen(FunctionCodegen.java:136)
        at org.jetbrains.kotlin.codegen.MemberCodegen.genSimpleMember(MemberCodegen.java:197)
        at org.jetbrains.kotlin.codegen.PackagePartCodegen.generateBody(PackagePartCodegen.java:98)
        at org.jetbrains.kotlin.codegen.MemberCodegen.generate(MemberCodegen.java:129)
        at org.jetbrains.kotlin.codegen.PackageCodegenImpl.generateFile(PackageCodegenImpl.java:149)
        at org.jetbrains.kotlin.codegen.PackageCodegenImpl.generate(PackageCodegenImpl.java:70)
        at org.jetbrains.kotlin.codegen.DefaultCodegenFactory.generatePackage(CodegenFactory.kt:77)
        at org.jetbrains.kotlin.codegen.DefaultCodegenFactory.generateModule(CodegenFactory.kt:62)
        at org.jetbrains.kotlin.codegen.KotlinCodegenFacade.compileCorrectFiles(KotlinCodegenFacade.java:35)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.generate(KotlinToJVMBytecodeCompiler.kt:595)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli(KotlinToJVMBytecodeCompiler.kt:211)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinToJVMBytecodeCompiler.compileModules$cli$default(KotlinToJVMBytecodeCompiler.kt:154)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:169)
        at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.doExecute(K2JVMCompiler.kt:52)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:88)
        at org.jetbrains.kotlin.cli.common.CLICompiler.execImpl(CLICompiler.kt:44)
        at org.jetbrains.kotlin.cli.common.CLITool.exec(CLITool.kt:98)
        at org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.runCompiler(IncrementalJvmCompilerRunner.kt:386)
        at org.jetbrains.kotlin.incremental.IncrementalJvmCompilerRunner.runCompiler(IncrementalJvmCompilerRunner.kt:110)
        at org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compileIncrementally(IncrementalCompilerRunner.kt:286)
        at org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compileImpl$rebuild(IncrementalCompilerRunner.kt:99)
        at org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compileImpl(IncrementalCompilerRunner.kt:114)
        at org.jetbrains.kotlin.incremental.IncrementalCompilerRunner.compile(IncrementalCompilerRunner.kt:74)
        at org.jetbrains.kotlin.daemon.CompileServiceImplBase.execIncrementalCompiler(CompileServiceImpl.kt:607)
        at org.jetbrains.kotlin.daemon.CompileServiceImplBase.access$execIncrementalCompiler(CompileServiceImpl.kt:96)
        at org.jetbrains.kotlin.daemon.CompileServiceImpl.compile(CompileServiceImpl.kt:1659)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at java.rmi/sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:359)
        at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:200)
        at java.rmi/sun.rmi.transport.Transport$1.run(Transport.java:197)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.rmi/sun.rmi.transport.Transport.serviceCall(Transport.java:196)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:562)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:796)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:677)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.rmi/sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:676)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalStateException: Couldn't obtain compiled function body for @kotlin.SinceKotlin @kotlin.time.ExperimentalTime @kotlin.internal.InlineOnly public inline fun kotlin.time.Duration.toJavaDuration(): java.time.Duration defined in kotlin.time[DeserializedSimpleFunctionDescriptor@3fc0dd9f]
        at org.jetbrains.kotlin.codegen.inline.InlineCodegen$Companion.getCompiledMethodNodeInner(InlineCodegen.kt:596)
        at org.jetbrains.kotlin.codegen.inline.InlineCodegen$Companion.createInlineMethodNode$backend(InlineCodegen.kt:549)
        at org.jetbrains.kotlin.codegen.inline.InlineCodegen.performInline(InlineCodegen.kt:140)
        ... 106 more

It's caused by the following descriptor:

        <dependency>
            <groupId>org.jetbrains.kotlin</groupId>
            <artifactId>kotlin-stdlib</artifactId>
            <version>${version.kotlin}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.jetbrains.kotlin</groupId>
            <artifactId>kotlin-reflect</artifactId>
            <version>${version.kotlin}</version>
            <scope>compile</scope>
        </dependency>

kotlin-stdlib is correctly defined as provided, but kotlin-reflect for some reason is defined as compile explicitly, which is weird. This was introduced in the commit: https://github.com/FasterXML/jackson-module-kotlin/commit/eedfb16233ddaca7d2ad1798bc9b1bfbd9da7a9a#diff-9c5fb3d1b7e3b0f54bc5c4182965c4fe1f9023d449017cece3005d3f90e8e4d8R89 before this dependency was scoped as provided.

It forces us to have the following configuration in our project:

    implementation "org.jetbrains.kotlin:kotlin-reflect"
    implementation ("com.fasterxml.jackson.module:jackson-module-kotlin")
    {
        exclude group: 'org.jetbrains.kotlin', module: 'kotlin-reflect'
    }

Expected behavior This module should

k163377 commented 1 year ago

Unfortunately, Kotlin 1.4 has been deprecated in the absence of the maintainer. Therefore, we do not plan to make any modifications to support Kotlin 1.4. We also plan to modify the documentation in the near future. #632

In order to avoid similar problems, our policy will be to use the most recent patch version within the smallest minor version that has not been deprecated.

Spikhalskiy commented 1 year ago

@k163377 the issues with Kotlin 1.4 was actually resolved in https://github.com/FasterXML/jackson-module-kotlin/pull/557 And the current state is fully compatible with Kotlin 1.4, we run Kotlin 1.4 pipelines with the new releases of Jackson as a part of Temporal CI/CD.

Unfortunately, Kotlin 1.4 has been deprecated in the absence of the maintainer.

I did take care of Kotlin 1.4 and can do it for some time.

In order to avoid similar problems, our policy will be to use the most recent patch version within the smallest minor version that has not been deprecated.

I personally don't think it's a very user-friendly version. Even JetBrains officially update and support the last three minor Kotlin releases.

We also plan to modify the documentation in the near future.

Not just the documentation, but the target will need to change. If Kotlin 1.4 is officially not supported, there should be no binary compatibility with Kotlin 1.4 (not being able to run at all is better that running incorrectly)

k163377 commented 1 year ago

@Spikhalskiy I am reading and writing English by machine translation, so please forgive me if I may have misunderstood or misrepresented something.

I personally don't think it's a very user-friendly version. Even JetBrains officially update and support the last three minor Kotlin releases.

For example, we currently use within Kotlin 1.5 and intend to use Kotlin 1.6 when Kotlin 1.5 is deprecated (i.e. Kotlin 1.9 is released). Is this policy not user-friendly?

Not just the documentation, but the target will need to change. If Kotlin 1.4 is officially not supported, there should be no binary compatibility with Kotlin 1.4 (not being able to run at all is better that running incorrectly)

To be precise, we were going to update the documentation and remove the tests in CI regarding Kotlin 1.4. Are you saying that this work is not enough?

I noticed that it would be preferable to also remove the code for compatibility with Kotlin 1.4, is this what you are referring to? https://github.com/FasterXML/jackson-module-kotlin/blob/7493f7d3d4fc9ea52b1dfc2adc781f4465ab4fdc/src/main/kotlin/com/fasterxml/ jackson/module/kotlin/KotlinAnnotationIntrospector.kt#L68

Spikhalskiy commented 1 year ago

For example, we currently use within Kotlin 1.5 and intend to use Kotlin 1.6 when Kotlin 1.5 is deprecated (i.e. Kotlin 1.9 is released). Is this policy not user-friendly?

This sounds good. Supporting [1.5,) when 1.8 is the latest released version is good enough!

To be precise, we were going to update the documentation and remove the tests in CI regarding Kotlin 1.4. Are you saying that this work is not enough?

Correct. I think we should make jackson-module-kotlin BINARY incompatible with Kotlin 1.4. So users get a build-time error if they try to use jackson-module-kotlin + Kotlin 1.4.

This can be achieved by building against Kotlin 1.6. Because Kotlin supports binary compatibility with one minor version down. So building against Kotlin 1.6 brings compatibility with 1.5, but not 1.4. Reference: https://kotlinlang.org/docs/kotlin-evolution.html#evolving-the-binary-format

Preferably (but we can't guarantee it), the binary format is mostly forwards compatible with the next feature release, but not later ones (in the cases when new features are not used, e.g. 1.3 can understand most binaries from 1.4, but not 1.5).

Spikhalskiy commented 1 year ago

This specific issue should be solved by declaring kotlin-reflect dependency as optional. Deprecation of Kotlin 1.4 will not solve it. Imagine the user having Kotlin 1.8 stdlib and jackson-module-kotlin bringing kotlin-reflect 1.5 in. I will submit a PR for it later. See how a similar situation with kotlin-reflect solved in micrometer: https://github.com/micrometer-metrics/micrometer/blob/main/micrometer-core/build.gradle#L86

nmck257 commented 1 year ago

Hey @Spikhalskiy -- have you had a chance to try this yet?

This specific issue should be solved by declaring kotlin-reflect dependency as optional. Deprecation of Kotlin 1.4 will not solve it. Imagine the user having Kotlin 1.8 stdlib and jackson-module-kotlin bringing kotlin-reflect 1.5 in. I will submit a PR for it later.

I encountered the same issue and imagined the same solution. If you're busy, I can try it myself, but if you already tried it and learned something, can you share?