Unable to serialize closed polymorphic structure with generic type in a wrapper

[rocketraman]

I have a closed polymorphic structure like this:

@Serializable
sealed class Foo<out T> {
  @Serializable
  data class Bar<out T>(
    val value: T?,
  ): Foo<T>()
}

@Serializable
data class Whatever(val foo: Foo<Boolean>)

Json.encodeToString(Whatever(Foo.Bar(true))) produces the error:

Exception in thread "main" kotlinx.serialization.SerializationException: Class 'Boolean' is not registered for polymorphic serialization in the scope of 'Any'.
Mark the base class as 'sealed' or register the serializer explicitly.
(edited)

Note that Json.encodeToString(Foo.Bar(true)) works, its only when trying to serialize Whatever we see the failure.

Environment

• Kotlin version: 1.5.31
• Library version: 1.3.1
• Kotlin platforms: JVM, JS
• Gradle version: 7.3
• IDE version (if bug is related to the IDE): 2021.3, but not IDE specific
• Other relevant context: Linux, JDK Adoptium 17

[rocketraman]

It appears a workaround is to bound the type of T using another sealed class hierarchy e.g.:

@Serializable
sealed class ValueHolder<T> {
  abstract val value: T?
}

@Serializable
data class BooleanValueHolder(override val value: Boolean?): ValueHolder<Boolean>()

@Serializable
sealed class Foo<T> {
  @Serializable
  data class Bar<T>(
    val value: ValueHolder<T>,
  ): Foo<T>()
}

@Serializable
data class Whatever(val foo: Foo<Boolean>)

// works!
println(Json.encodeToString(Whatever(Foo.Bar(BooleanValueHolder(true)))))

But this really seems unnecessary.

[sandwwraith]

It is indeed the limitation of sealed classes: they do not accept generic type parameters serializers and resort to polymorphism instead. I think the second example is working by accident

[rocketraman]

In that case what is the correct way to represent such a structure that is compatible with serialization?

[rocketraman]

I think the second example is working by accident

When you say "second example" did you mean this is working by accident:

1) Json.encodeToString(Foo.Bar(true))

or this is working by accident:

2) "It appears a workaround is to bound the type of T using another sealed class hierarchy"

I want to be certain my data model doesn't rely on an accidental feature.

[pdvrieze]

I would say that the second example isn't working by accident, but by design as BooleanValueType does not have type parameters and can be validly serialized. But going to the broader question, type parameters on a serializable type will lead to a parameterised serializer where the serializer for the type is provided as constructor parameter (this is what the generator expects).

The way that serialization of a sealed type works is that the parent serializer exposes the serializer for each child type. At the point these serializers are created (for the generic type, not a single instance) there is no concrete type bound to the type variable, as such the supertype is used (Any in this case) with polymorphic serialization, or in the case the supertype is a sealed type with sealed serialization (which is a variant on polymorphic).

Of course you can create a custom serializer for Bar that does something smarter, but otherwise you have to deal with the polymorphic nature of the parameter.

[rocketraman]

The way that serialization of a sealed type works is that the parent serializer exposes the serializer for each child type. At the point these serializers are created (for the generic type, not a single instance) there is no concrete type bound to the type variable, as such the supertype is used (Any in this case) with polymorphic serialization, or in the case the supertype is a sealed type with sealed serialization (which is a variant on polymorphic).

@pdvrieze I've tried many things to tell kotlinx-serialization how to deal with the type parameter, but unfortunately have not been able to make anything work.

For example (all of the above based on the sealed class definition and sample input given in the OP):

val format = Json { serializersModule = SerializersModule {
  polymorphic(Any::class) {
    subclass(Boolean::class, Boolean.serializer())
  }
} }

and

val format = Json { serializersModule = SerializersModule {
  polymorphic(Foo::class) {
    subclass(Foo.Bar::class, Foo.Bar.serializer(Boolean.serializer()) as kotlinx.serialization.KSerializer<Foo.Bar<*>>)
  }
} }

Am I missing something about how t deal with the type parameter properly, or is this in fact a design gap in kotlinx.serialization as initialy stated by @sandwwraith ?

[rocketraman]

Even with my ValueHolder workaround I am running into this error, for which I see multiple related issues and fixes (https://github.com/Kotlin/kotlinx.serialization/issues/1584, https://github.com/Kotlin/kotlinx.serialization/issues/1770, https://github.com/Kotlin/kotlinx.serialization/issues/1646), but no workarounds -- I can't upgrade to 1.6.0 because Compose does not yet support it:

e: java.lang.IllegalStateException: Not found Idx for public com.xyz.model/Foo|null[0]
        at org.jetbrains.kotlin.backend.common.serialization.IrFileDeserializer.loadTopLevelDeclarationProto(IrFileDeserializer.kt:48)
        at org.jetbrains.kotlin.backend.common.serialization.IrFileDeserializer.deserializeDeclaration(IrFileDeserializer.kt:39)
        at org.jetbrains.kotlin.backend.common.serialization.FileDeserializationState.deserializeAllFileReachableTopLevel(IrFileDeserializer.kt:139)
        at org.jetbrains.kotlin.backend.common.serialization.ModuleDeserializationState.deserializeReachableDeclarations(BasicIrModuleDeserializer.kt:172)
        at org.jetbrains.kotlin.backend.common.serialization.BasicIrModuleDeserializer.deserializeReachableDeclarations(BasicIrModuleDeserializer.kt:148)
        at org.jetbrains.kotlin.backend.common.serialization.KotlinIrLinker.deserializeAllReachableTopLevels(KotlinIrLinker.kt:102)
        at org.jetbrains.kotlin.backend.common.serialization.KotlinIrLinker.findDeserializedDeclarationForSymbol(KotlinIrLinker.kt:121)
        at org.jetbrains.kotlin.backend.common.serialization.KotlinIrLinker.getDeclaration(KotlinIrLinker.kt:159)
        at org.jetbrains.kotlin.ir.util.ExternalDependenciesGeneratorKt.getDeclaration(ExternalDependenciesGenerator.kt:60)
        at org.jetbrains.kotlin.ir.util.ExternalDependenciesGenerator.generateUnboundSymbolsAsDependencies(ExternalDependenciesGenerator.kt:47)
        at org.jetbrains.kotlin.ir.backend.js.KlibKt.loadIr(klib.kt:350)
        at org.jetbrains.kotlin.ir.backend.js.KlibKt.loadIr$default(klib.kt:232)
        at org.jetbrains.kotlin.ir.backend.js.CompilerKt.compile(compiler.kt:97)
        at org.jetbrains.kotlin.ir.backend.js.CompilerKt.compile$default(compiler.kt:42

[rocketraman]

The comments by @christofvanhove at https://github.com/Kotlin/kotlinx.serialization/issues/1252 are helpful.

[StarGuardian]

I hope my solutions is helpful: https://github.com/Kotlin/kotlinx.serialization/issues/1252#issuecomment-1780935921