Moshi fails to process Kotlin 1.6 metadata

[max-kammerer]

Caused by: java.lang.IllegalStateException: Could not parse metadata! This should only happen if you're using Kotlin <1.1.
    at com.squareup.moshi.kotlinpoet.metadata.KotlinPoetMetadata.readKotlinClassMetadata(KotlinPoetMetadata.kt:89)
    at com.squareup.moshi.kotlinpoet.metadata.KotlinPoetMetadata.toImmutableKmClass(KotlinPoetMetadata.kt:119)
    at com.squareup.moshi.kotlin.codegen.MoshiCachedClassInspector.toImmutableKmClass(MoshiCachedClassInspector.kt:37)
    at com.squareup.moshi.kotlin.codegen.MetadataKt.targetType(metadata.kt:139)
    at com.squareup.moshi.kotlin.codegen.JsonClassCodegenProcessor.adapterGenerator(JsonClassCodegenProcessor.kt:148)

Project with reproduction test-moshi-kotlin16.zip

[ZacSweers]

Thanks, we can update but not sure when we'll have a release. Wish this API would offer something stable that we don't need to shade

[ZacSweers]

hmm, we are already on kotlinx-metadata 0.3.0. Unfortunately I'm not too well-versed in maven to know what's possibly wrong with that build :/

[realdadfish]

Happens to me in a Gradle build as well, as soon as I set the languageVersion to "1.6" (to make use of the sealed when preview in Kotlin 1.5.30):

Caused by: java.lang.IllegalStateException: Could not parse metadata! This should only happen if you're using Kotlin <1.1.
        at com.squareup.moshi.kotlinpoet.metadata.KotlinPoetMetadata.readKotlinClassMetadata(KotlinPoetMetadata.kt:89)
        at com.squareup.moshi.kotlinpoet.metadata.KotlinPoetMetadata.toImmutableKmClass(KotlinPoetMetadata.kt:119)
        at com.squareup.moshi.kotlin.codegen.MoshiCachedClassInspector.toImmutableKmClass(MoshiCachedClassInspector.kt:37)
        at com.squareup.moshi.kotlin.codegen.MetadataKt.targetType(metadata.kt:139)
        at com.squareup.moshi.kotlin.codegen.JsonClassCodegenProcessor.adapterGenerator(JsonClassCodegenProcessor.kt:148)
        at com.squareup.moshi.kotlin.codegen.JsonClassCodegenProcessor.process(JsonClassCodegenProcessor.kt:114)
        at org.jetbrains.kotlin.kapt3.base.incremental.IncrementalProcessor.process(incrementalProcessors.kt:90)
        at org.jetbrains.kotlin.kapt3.base.ProcessorWrapper.process(annotationProcessing.kt:188)
        at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor(JavacProcessingEnvironment.java:980)
        ... 41 more

And yes, kotlinx-metadata is already silently updated to 0.3.0 for me as well and there doesn't seem to be a newer version of this library available.

+--- com.squareup.moshi:moshi-kotlin-codegen -> 1.12.0
|    +--- com.squareup.moshi:moshi:1.12.0
|    |    +--- com.squareup.okio:okio:2.10.0
|    |    |    +--- org.jetbrains.kotlin:kotlin-stdlib:1.4.20 -> 1.5.30 (*)
|    |    |    \--- org.jetbrains.kotlin:kotlin-stdlib-common:1.4.20 -> 1.5.30
|    |    \--- org.jetbrains.kotlin:kotlin-stdlib:1.4.31 -> 1.5.30 (*)
|    +--- org.jetbrains.kotlin:kotlin-reflect:1.4.31 -> 1.5.30
|    |    \--- org.jetbrains.kotlin:kotlin-stdlib:1.5.30 (*)
|    +--- com.squareup:kotlinpoet:1.8.0
|    |    +--- org.jetbrains.kotlin:kotlin-reflect:1.4.31 -> 1.5.30 (*)
|    |    \--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.31 -> 1.5.30 (*)
|    +--- com.squareup:kotlinpoet-classinspector-elements:1.8.0
|    |    +--- com.squareup:kotlinpoet-metadata-specs:1.8.0
|    |    |    +--- com.squareup:kotlinpoet:1.8.0 (*)
|    |    |    +--- com.squareup:kotlinpoet-metadata:1.8.0
|    |    |    |    +--- org.jetbrains.kotlinx:kotlinx-metadata-jvm:0.2.0 -> 0.3.0 (*)
|    |    |    |    \--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.31 -> 1.5.30 (*)
|    |    |    \--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.31 -> 1.5.30 (*)
|    |    +--- com.google.guava:guava:29.0-jre (*)
|    |    \--- org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.31 -> 1.5.30 (*)

This seems to be the upstream tracking bug: https://youtrack.jetbrains.com/issue/KT-47945

[max-kammerer]

NB: I've deleted ~/.m2/repository/org/jetbrains/kotlinx/kotlinx-metadata-jvm folder and after project reexecution there is only 0.2.0 version in it (0.3.0 should be compatible with Kotlin 1.6)

[max-kammerer]

mvn dependency:tree with moshi-kotlin-coden added to build dependencies shows that moshi ships obsolete dependency to kotlinx-metadata-jvm through kotlinpoet:

[INFO] +- com.squareup.moshi:moshi-kotlin-codegen:jar:1.12.0:compile
[INFO] |  +- com.squareup:kotlinpoet:jar:1.8.0:compile
[INFO] |  +- com.squareup:kotlinpoet-classinspector-elements:jar:1.8.0:compile
[INFO] |  |  +- com.squareup:kotlinpoet-metadata-specs:jar:1.8.0:compile
[INFO] |  |  |  \- com.squareup:kotlinpoet-metadata:jar:1.8.0:compile
[INFO] |  |  |     \- org.jetbrains.kotlinx:kotlinx-metadata-jvm:jar:0.2.0:compile

Full tree

[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ test-lombok-with-kapt ---
[INFO] org.jetbrains.kotlin:test-lombok-with-kapt:jar:1.0-SNAPSHOT
[INFO] +- org.jetbrains.kotlin:kotlin-stdlib:jar:1.5.30:compile
[INFO] |  +- org.jetbrains:annotations:jar:13.0:compile
[INFO] |  \- org.jetbrains.kotlin:kotlin-stdlib-common:jar:1.5.30:compile
[INFO] +- com.squareup.moshi:moshi-kotlin:jar:1.12.0:compile
[INFO] |  +- com.squareup.moshi:moshi:jar:1.12.0:compile
[INFO] |  |  \- com.squareup.okio:okio:jar:2.10.0:compile
[INFO] |  \- org.jetbrains.kotlin:kotlin-reflect:jar:1.4.31:compile
[INFO] +- com.squareup.moshi:moshi-kotlin-codegen:jar:1.12.0:compile
[INFO] |  +- com.squareup:kotlinpoet:jar:1.8.0:compile
[INFO] |  +- com.squareup:kotlinpoet-classinspector-elements:jar:1.8.0:compile
[INFO] |  |  +- com.squareup:kotlinpoet-metadata-specs:jar:1.8.0:compile
[INFO] |  |  |  \- com.squareup:kotlinpoet-metadata:jar:1.8.0:compile
[INFO] |  |  |     \- org.jetbrains.kotlinx:kotlinx-metadata-jvm:jar:0.2.0:compile
[INFO] |  |  \- com.google.guava:guava:jar:29.0-jre:compile
[INFO] |  |     +- com.google.guava:failureaccess:jar:1.0.1:compile
[INFO] |  |     +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
[INFO] |  |     +- com.google.code.findbugs:jsr305:jar:3.0.2:compile
[INFO] |  |     +- org.checkerframework:checker-qual:jar:2.11.1:compile
[INFO] |  |     +- com.google.errorprone:error_prone_annotations:jar:2.3.4:compile
[INFO] |  |     \- com.google.j2objc:j2objc-annotations:jar:1.3:compile
[INFO] |  +- org.ow2.asm:asm:jar:7.1:compile
[INFO] |  +- com.google.auto.service:auto-service-annotations:jar:1.0-rc7:compile
[INFO] |  +- net.ltgt.gradle.incap:incap:jar:0.3:compile
[INFO] |  \- org.jetbrains.kotlin:kotlin-stdlib-jdk8:jar:1.4.31:compile
[INFO] |     \- org.jetbrains.kotlin:kotlin-stdlib-jdk7:jar:1.4.31:compile
[INFO] \- org.projectlombok:lombok:jar:1.18.20:provided

[ZacSweers]

We shade both kotlinx-metadata and kotlinpoet's extensions for it, I'm not sure why those would show up as dependencies https://github.com/square/moshi/blob/master/kotlin/codegen/build.gradle.kts

[max-kammerer]

As I see kotlinx-metadata was updated to 0.3.0 on May 7 by this commit But last moshi release was in April. Same dependency in poet was updated in 1.9.0 release

[ZacSweers]

Thanks. I'm going to close this as it's already fixed on master, we'll do a release by or around the 1.6 release

[realdadfish]

Is there a specific reason not to release this earlier, i.e. now? As I said, this blocks the Kotlin 1.5.31 update for people like me who want to opt-in for sealed whens.

[max-kammerer]

+1 for earlier release to unblock Kotlin 1.6.0 EAPers that use moshi

[ZacSweers]

Because we're not ready for another release yet (we have more in the pipeline for 1.13.0) and you can easily unblock yourselves by building from source or use snapshots if you need it sooner.

[ZacSweers]

As I said, this blocks the Kotlin 1.5.31 update for people like me who want to opt-in for sealed whens.

This is not true. Sealed whens are gated on the -progressive flag, which does not change the language version.

[realdadfish]

As I said, this blocks the Kotlin 1.5.31 update for people like me who want to opt-in for sealed whens.

This is not true. Sealed whens are gated on the -progressive flag, which does not change the language version.

Oh, this is news, because the release notes state otherwise:

To enable this feature in Kotlin 1.5.30, use language version 1.6.

(https://kotlinlang.org/docs/whatsnew1530.html#exhaustive-when-statements-for-sealed-and-boolean-subjects)

[ZacSweers]

progressive enables warnings and you can enable warnings as errors. 1.6 target language feature makes it an error out of the box

[realdadfish]

OK, I misread this then, I thought it was required to have it warn about it at all in 1.5 even. Thanks for the clarification!

[ZacSweers]

The ideal scenario would be that Jetbrains make this an actual stable API (https://youtrack.jetbrains.com/issue/KT-48011) or didn't impose this reading limit since it's technically just protobufs under the hood and should be compatible to read.

[evitulano]

Thanks. I'm going to close this as it's already fixed on master, we'll do a release by or around the 1.6 release

Koltin 1.6 is out, it's possible to make a new release with the issue fixed? Thanks

[DavideSorcelli]

Also reproduced in my multi-module project, based on clean architecture. Presentation layer depends on Framework layer that has all retrofit/okhttp/moshi dependencies (including Codegen). I'm using hilt di modules to provide retrofit clients in Framework layer.

After updating kotlin to 1.6.0 I'm getting this error at build time:

Caused by: java.lang.IllegalStateException: Could not parse metadata! This should only happen if you're using Kotlin <1.1. at com.squareup.moshi.kotlinpoet.metadata.KotlinPoetMetadata.readKotlinClassMetadata(KotlinPoetMetadata.kt:89) at com.squareup.moshi.kotlinpoet.metadata.KotlinPoetMetadata.toImmutableKmClass(KotlinPoetMetadata.kt:119) at com.squareup.moshi.kotlin.codegen.MoshiCachedClassInspector.toImmutableKmClass(MoshiCachedClassInspector.kt:37) at com.squareup.moshi.kotlin.codegen.MetadataKt.targetType(metadata.kt:139) at com.squareup.moshi.kotlin.codegen.JsonClassCodegenProcessor.adapterGenerator(JsonClassCodegenProcessor.kt:148)

The only way to get it working is to revert "org.jetbrains.kotlin:kotlin-gradle-plugin" dep from 1.6.0 to 1.5.31.

So i hope Moshi team could fix this as fast as possible, because this issue won't let me use the latest version of kotlin.

[malloth]

Any way around this bug? I can not wait until new Moshi is released (it takes forever to prepare a simple bugfix for 1.12.1)

[iainism]

Other projects are now starting to mandate the use of Kotlin 1.6.0 to be able to remain current (get bug fixes, new features, etc.), e.g.: https://kotest.io/docs/changelog.html#kotlin-16-is-now-the-minimum-supported-version

What is the timeline for a release containing the fix please?

[ImaginativeShohag]

Jetpack Compose 1.1.0-beta04 now needed Kotlin 1.6.0. Moshi needs to publish a fix soon.

[malloth]

I'm thinking about switching to Kotlin Serialization because Moshi team is not being able to release a quick hotfix. So lame!

[jakoss]

@malloth Go ahead. Or maybe you can just do this "quick hotfix" by yourself on fork and use your own release? You can release it to public and be the hero we all deserve! Either way - stop making GH toxic place

[gmk57]

...or just use a snapshot version.

[lsuski]

I think that some frustration is understandable on both sides of this issue. People are blocked and don't know how long it will take, there is no hint about that. On the other side moshi team is bombarded with questions like "When are you going to release this?". This is open source so noone of us pays moshi team for maintaing this repo, everyone can release this on their own from fork. However this require some effort and time and before doing this it would be nice to know if it makes sense. If there will be an official release in few days, maybe 2 weeks then personally I will wait. Can we have such information? Everyone can then make his own decision what to do next and not spam this thread, release fork, wait or migrate to something else.

[jakoss]

@lsuski Totally agree with that. I have problem with the attitude and unnecessary insults, but I do understand the problem.

No amount of frustration justifies being toxic.

[nokite]

Going off-topic again, I'll venture a speculation that Square would prefer to get more support from Jetbrains with a more open contract, so that Moshi (and other libs) don't have to be coded for each specific Kotlin version, which is more work and obviously breaks. At the same time Jetbrains might not care enough, as they have an alternative solution - Kotlin Serialization. I hope neither of them is using the current tension for leverage.

Edit: Better ignore this, not a good idea to assume or imply things, and I was in a bad mood when writing it 😀

[gmk57]

@nokite Well, Zac Sweers said it in plain text above, and everybody could star the issue on YouTrack. It currently has "Priority: Major" and "State: In Progress", which gives me some hope that things will improve eventually.

[ZacSweers]

Hey folks, we owe a bit of an explanation. Apologies for the radio silence.

• Yes, we are late on a release here. Square folks are combing through PRs to catch up (it's been mostly me since the last release, I don't work at Square nor do I control releases). They are almost done, but I can't offer any concrete ETA other than "soon".
• Yes this kotlinx-metadata stability issue is a problem. We, perhaps ironically in this case, shade any unstable APIs we use in order to guarantee future compatibility. I filed the issue above after talking with JB folks and hopeful it'll go stable soon. Unfortunately this is sort of similar to ASM for Java where it can't make guarantees about being able to read future versions, only backward compatible releases. When it's stable at least, folks will be able to unblock locally by just forcing a newer version of it since we won't have to shade it anymore.
• The next Moshi release will use kotlinx-metadata 0.4.0, which will support both Kotlin 1.6 as well as Kotlin 1.7 later down the line (it guarantees full backward compatibility + one future release), so that will buy more runway.
• We close issues when they are fixed on main, not when they are released.

As far as being blocked - as others have mentioned already, you aren't truly blocked unless you're maintaining other published libraries that depend on this. Here are some patterns we use to avoid being bottlenecked on external dependencies.

1. Download the latest snapshot jars (link in the README) or build from source (see RELEASING.md or ci.yml to see how we build artifacts)
2. You can check these into your repo or upload them to an internal artifactory/nexus/etc instance (change the version in the pom.xml file if you do upload). We do this with Moshi in our Slack repo today.

If you check the jars into your repo, you can put them in a standalone project like so

// In a standalone project like `tmp-moshi`
dependencies {
    implementation(files('libs/moshi.jar'))
}

Then add this to your root build.gradle file to substitute remote Moshi dependencies with that project

subprojects {
  configurations.configureEach {
    resolutionStrategy {
        dependencySubstitution {
            substitute(module("com.squareup.moshi:moshi")).apply {
              using(project("tmp-moshi")) // Your local project holding the moshi jars
              because("Moshi 1.13.0 is not released yet, this is a local cut from HEAD")
            }
            // Repeat for any other moshi artifacts you use (code gen, kotlin reflect, adapters, etc)
        }
    }
  }
}

If you upload jars to an artifactory, simply just use a newer version like 1.13.0-yourcompany and force that version instead. This whole process takes me around 30 seconds to do when we take early cuts of external libraries, so hope it's as easy elsewhere 🤞.

I hope this is helpful and thanks for bearing with us.

[swankjesse]

1.13.0 is out. Thanks for your patience.

[Raykud]

I get Records requires ASM8 when I use 1.13.0 and having Java jdk-11.0.12.jdk

[uberchilly]

I get Records requires ASM8 when I use 1.13.0 and having Java jdk-11.0.12.jdk

This I think is issue with firebase in combination with Moshi and not Moshi itself

[ZacSweers]

Correct, please take up issues with other tools on their respective issue trackers. One thing this release has revealed is some tools incorrectly read class files out of META-INF, which is a bug on their side and not something Moshi can fix.

[aman1sr]

for me it worked when i upgraded: Moshi -> 1.14.0 Kotlin -> 1.7.0