Add ability to decode to a raw ByteArray

[anpez]

Is your feature request related to a problem? Please describe. I'm trying to parse an array of polymorphic items. With one single item, I can parse the "header" of the item, decide which type I should decode into, and then perform one more decode with the full object. With an array of different items this doesn't seem to be possible.

Describe the solution you'd like In golang, there's a json.RawMessage or, specifically in the msgpack implementation I'm using, a codec.Raw type, which lets partially decode an object, and then parse the rest of the object once the concrete type is known.

Describe alternatives you've considered I've tried a custom KSerializer and decoding into a ByteArray/String with rawCompatibility enabled, but that didn't seem to work. Also, looking at the code itself, it doesn't look like it's supported right now.

[esensar]

Is it possible to achieve something similar with MsgPackDynamicSerializer right now? I will look into this. Could you provide an example code of what you are trying to achieve?

[anpez]

Sorry for the delayed response, I've been trying to use MsgPackDynamicSerializer but doesn't seem to work for me, but maybe it's skill issue. What I'm trying to achieve is to have something like this:

sealed class BaseCommand {
    val id: Long

    data class MultipleCommands(override val id: Long, val commands: List<BaseCommand>): BaseCommand()
    data class WorkspaceList(override val id: Long, val workspaces: List<Workspace>): BaseCommand()
    data class ACK(override val id: Long): BaseCommand()
}
mapOf(
    "id" to 0L,
    "type" to MsgTypeMultipleCommands.value,
    "commands" to listOf(
        mapOf(
            "id" to 1L,
            "type" to 1000,
            "data" to emptyList<WorkspaceRemote>()
        ),
        mapOf(
            "id" to 2L,
            "type" to 2000,
            "data" to emptyList<ChannelRemote>()
        ),
        mapOf(
            "id" to 3L,
            "type" to 3000,
        ),
    ),
)

to be parsed into a MultipleCommands data class that contains a list of different commands, all of which have some common fields, but others are specific to the type. Does it make sense?

edit: to clarify, what I'm intending to do with the "raw" thing, is to be able to do something like this:

@Serializable
data class BaseCommand(
val id: Long,
val type: Short,
@RawMsgPack val data: ByteArray,
)

[esensar]

Sorry for the delayed response, I've been trying to use MsgPackDynamicSerializer but doesn't seem to work for me, but maybe it's skill issue. What I'm trying to achieve is to have something like this:

sealed class BaseCommand {
    val id: Long

    data class MultipleCommands(override val id: Long, val commands: List<BaseCommand>): BaseCommand()
    data class WorkspaceList(override val id: Long, val workspaces: List<Workspace>): BaseCommand()
    data class ACK(override val id: Long): BaseCommand()
}
mapOf(
    "id" to 0L,
    "type" to MsgTypeMultipleCommands.value,
    "commands" to listOf(
        mapOf(
            "id" to 1L,
            "type" to 1000,
            "data" to emptyList<WorkspaceRemote>()
        ),
        mapOf(
            "id" to 2L,
            "type" to 2000,
            "data" to emptyList<ChannelRemote>()
        ),
        mapOf(
            "id" to 3L,
            "type" to 3000,
        ),
    ),
)

to be parsed into a MultipleCommands data class that contains a list of different commands, all of which have some common fields, but others are specific to the type. Does it make sense?

edit: to clarify, what I'm intending to do with the "raw" thing, is to be able to do something like this:

@Serializable
data class BaseCommand(
val id: Long,
val type: Short,
@RawMsgPack val data: ByteArray,
)

Sorry for the late response. I have been looking into ways to achieve that with just the official polymorphism guide (https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/polymorphism.md), but no luck. I wanted to avoid building something completely new in the library, especially since something like this would make more sense in kotlinx.serialization in general, rather than just this MsgPack implementation.

It looks like by default type field is added to serialized objects which contains class name to be used when serializing/deserializing. It is possible to add customization option for that, but it would still have to be a string (and not an integer type) unfortunately.