Combining `SealedClassSerializer`s

[Whathecode]

What is your use-case and why do you need this feature?

I have a base, non-sealed, interface, from which sealed classes extend. At runtime (this is a generic framework), I need to be able to create a serializer which can handle all subclasses of multiple sealed classes that extend from this common interface.

My initial expectation was that I could instantiate a new SealedClassSerializer and pass all subclasses and subclassSerializers from the other serializers as constructor parameters. However, I found out that after instantiating SealedClassSerializer this information is no longer accessible (it is stored in a private class2Serializer field).

As an example, my specific use case: I have an IntegrationEvent interface and application services which each define the events they emit by defining a sealed class which extends from this interface. Application services can subscribe to events from other services. Serializing any event received from dependent services thus requires me to be able to deserialize any of the events defined as subclasses for each of the sealed event classes.

P.s. I understand SealedClassSerializer is an internal serialization API; I find myself accessing internal APIs quite frequently since I'm using kotlinx.serialization in quite elaborate use cases. I hope sharing these may inspire some of the internal APIs to be stabilized and made public, since I would argue these are valid use cases. For now, I don't mind using them as unstable APIs.

Describe the solution you'd like

Could subclasses and subclassSerializers, or a meaningful map thereof, be made public as readonly fields? Once you have access to this class, I don't think information hiding makes sense. In addition, maybe a factory method can be added which takes multiple other sealed class serializers to combine them.

I currently hack this using JVM reflection as workaround:

// Get event serializer capable of serializing published events, as well as events the service subscribes to.
val eventSerializers = dependentServices.plus( serviceKlass ).map { getEventSerializer( it ) }
// HACK: There is no public accessor to get the subclass serializers. There probably should be.
val subclassesField = SealedClassSerializer::class.java
    .getDeclaredField( "class2Serializer" )
    .apply { isAccessible = true }
val allSubclassSerializers = eventSerializers.flatMap {
    @Suppress( "UNCHECKED_CAST" )
    val serializers = subclassesField.get( it )
        as Map<KClass<IntegrationEvent<*>>, KSerializer<IntegrationEvent<*>>>
    serializers.toList()
}.toMap()
eventSerializer = SealedClassSerializer(
    IntegrationEvent::class.qualifiedName!!,
    IntegrationEvent::class,
    allSubclassSerializers.keys.toTypedArray(),
    allSubclassSerializers.values.toTypedArray()
)

[sandwwraith]

So basically you want something like that, right?

val myNewSerializer = SealedClassSerializer(sealedSerializer1.sealedSubclasses + sealedSerializer2.sealedSubclasses)

for combining two sealed hierarchies.

It sounds quite reasonable since we have an API to combine SerializersModules , but no API to combine sealed subclasses. Maybe we should even go further and provide some kind of converter/extractor from SealedClassSerializer to a module with its subclasses.

[tadfisher]

Ideally one wouldn't have to care whether the polymorphic subclass is sealed or not, and the runtime would add all concrete subclasses of the subtype without needing to combine modules. At least I can't think of a use case where one would not want this to happen automatically.

For example, I added a PolymorphicModuleBuilder.subclassesOf extension that does just that:

// HACK: We can't access the subclass serializers in SealedClassSerializer because
// they are stored in a private "class2Serializer" field. This exposes the field
// via reflection.
// Solution taken from: https://github.com/Kotlin/kotlinx.serialization/issues/1865
@OptIn(InternalSerializationApi::class)
internal actual object SealedClassSerializerPrivate {
    private val class2SerializersField = SealedClassSerializer::class.java
        .getDeclaredField("class2Serializer")
        .apply { isAccessible = true }

    @Suppress("UNCHECKED_CAST")
    actual fun <T : Any> class2Serializer(
        serializer: SealedClassSerializer<T>
    ): Map<KClass<out T>, KSerializer<out T>> =
        class2SerializersField.get(serializer) as Map<KClass<out T>, KSerializer<out T>>
}

@OptIn(InternalSerializationApi::class)
private fun <T : Any> findSubclassSerializers(
    subclass: KClass<out T>,
    serializer: KSerializer<out T>
): Map<KClass<out T>, KSerializer<out T>> = when (serializer) {
    is SealedClassSerializer<out T> -> SealedClassSerializerPrivate.class2Serializer(serializer)
        .flatMap { (subclass_, serializer_) ->
            findSubclassSerializers(subclass_, serializer_).toList()
        }
        .toMap()
    else -> mapOf(subclass to serializer)
}

@Suppress("UNCHECKED_CAST", "NOTHING_TO_INLINE")
@PublishedApi
internal inline fun <T> KSerializer<*>.cast(): KSerializer<T> = this as KSerializer<T>

fun <Base : Any, T : Base> PolymorphicModuleBuilder<Base>.subclassesOf(
    subclass: KClass<T>,
    serializer: KSerializer<T>
) {
    for ((subclass_, serializer_) in findSubclassSerializers(subclass, serializer)) {
        subclass(subclass_, serializer_.cast())
    }
}

inline fun <Base : Any, reified T : Base> PolymorphicModuleBuilder<Base>.subclassesOf(
    serializer: KSerializer<T>
): Unit = subclassesOf(T::class, serializer)

inline fun <Base : Any, reified T : Base> PolymorphicModuleBuilder<Base>.subclassesOf(
    clazz: KClass<T>
): Unit = subclassesOf(clazz, serializer())

And now one can register a sealed type as a subclass as long as it is serializable itself, as the following test demonstrates:

interface BaseInterface

@Serializable
sealed interface SealedInterface : BaseInterface {

    @Serializable
    object Object : SealedInterface
}

val json = Json {
    serializersModule = SerializersModule {
        polymorphic(BaseInterface::class) {
            subclassesOf(SealedInterface::class)
        }
    }
}

val encoded = json.encodeToString<BaseInterface>(SealedInterface.Object)
// -> {"type":"SealedInterface.Object"}
json.decodeFromString<BaseInterface>(encoded)
// -> SealedInterface.Object

[pdvrieze]

Looking at the fact that it is/should be possible to have custom serializers of SEALED or OPEN polymorphic kind this needs some significant thought in API. Normally Serializers don't allow querying the child serializers (neither directly, nor through the descriptor). This design makes it easier to write custom serializers, but if it is used mainly to get information out of "generated" serializers it would be good to be able to query those (as long as it doesn't make formats lazy).