IllegalStateException: Must call beginStructure() and use returned Decoder when using `@Contextual`

Describe the bug

yaml.decodeFromString() throws an IllegalStateException: Must call beginStructure() and use returned Decoder when using the @Contextual annotation on a property

Reproduction repo

import com.charleskorn.kaml.Yaml
import kotlinx.serialization.Contextual
import kotlinx.serialization.KSerializer
import kotlinx.serialization.Serializable
import kotlinx.serialization.descriptors.PrimitiveKind
import kotlinx.serialization.descriptors.PrimitiveSerialDescriptor
import kotlinx.serialization.descriptors.SerialDescriptor
import kotlinx.serialization.encodeToString
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.modules.SerializersModule
import java.time.LocalDate
import kotlin.test.Test
import kotlin.test.assertEquals

class ConfigTest {
    private val yaml: Yaml = Yaml(
        serializersModule = SerializersModule {
            contextual(LocalDate::class, LocalDateSerializer)
        },
    )

    @Test
    fun `Test serialization and deserialization`() {
        val config = Config(
            dateOfBirth = LocalDate.parse("1980-01-01"),
        )

        val yamlString = yaml.encodeToString(config)
        assertEquals(config, yaml.decodeFromString(Config.serializer(), yamlString))
    }
}

@Serializable
data class Config(
    @Contextual
    val dateOfBirth: LocalDate,
)

object LocalDateSerializer : KSerializer<LocalDate> {
    override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("LocalDate", PrimitiveKind.STRING)

    override fun serialize(encoder: Encoder, value: LocalDate) {
        encoder.encodeString(value.toString())
    }

    override fun deserialize(decoder: Decoder): LocalDate {
        val string = decoder.decodeString()
        return LocalDate.parse(string)
    }
}

Steps to reproduce

Run the test Test serialization and deserialization

Expected behaviour

The data class

@Serializable
data class Config(
    @Contextual
    val dateOfBirth: LocalDate,
)

is correctly encoded and decoded.

Actual behaviour

The data class Config is correctly encoded (dateOfBirth: "1980-01-01") but, when decoded it throws an IllegalStateException.

Version information

`"com.charleskorn.kaml:kaml:0.60.0"`

Any other information

Using

@Serializable
data class Config(
    @Serializable(with = LocalDateSerializer::class)
    val dateOfBirth: LocalDate,
)

instead of

@Serializable
data class Config(
    @Contextual
    val dateOfBirth: LocalDate,
)

works around the issue but it would be nice to be able to use the @Contextual annotation.

Have you followed the instructions in the error message? You need to modify LocalDateSerializer.serialize() to call beginStructure() as it suggests.

Hi @charleskorn,

I've been trying for about an hour to get a Serializer to work with Yaml and Json but no luck yet. Here's what’s weird: my original Serializer works perfectly when I use it kotlinx.serialization.json.Json so the issue seems to be Yaml specific:

import kotlinx.serialization.Contextual
import kotlinx.serialization.KSerializer
import kotlinx.serialization.Serializable
import kotlinx.serialization.descriptors.PrimitiveKind
import kotlinx.serialization.descriptors.PrimitiveSerialDescriptor
import kotlinx.serialization.descriptors.SerialDescriptor
import kotlinx.serialization.encodeToString
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.json.Json
import kotlinx.serialization.modules.SerializersModule
import java.time.LocalDate
import kotlin.test.Test
import kotlin.test.assertEquals

class ConfigTest {
    private val json: Json = Json {
        serializersModule = SerializersModule {
            contextual(LocalDate::class, LocalDateSerializer)
        }
    }

    @Test
    fun `Test serialization and deserialization`() {
        val config = Config(
            dateOfBirth = LocalDate.parse("1980-01-01"),
        )

        val jsonString = json.encodeToString(config)
        assertEquals(config, json.decodeFromString(Config.serializer(), jsonString))
    }
}

@Serializable
data class Config(
    @Contextual
    val dateOfBirth: LocalDate,
)

object LocalDateSerializer : KSerializer<LocalDate> {
    override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("LocalDate", PrimitiveKind.STRING)

    override fun serialize(encoder: Encoder, value: LocalDate) {
        encoder.encodeString(value.toString())
    }

    override fun deserialize(decoder: Decoder): LocalDate {
        val string = decoder.decodeString()
        return LocalDate.parse(string)
    }
}

Any idea why it's not playing nice with com.charleskorn.kaml.Yaml?

Hi, I looked at this more closely - from my perspective, the Yaml decoder incorrectly handles cases when a descriptor has SerialKind.CONTEXTUAL kind. Documentation for this kind says:

Represents an "unknown" type that will be known only at the moment of the serialization. ... To introspect descriptor of this kind, an instance of SerializersModule is required.

If I understood this correctly, we need to call SerializersModule.getContextualDescriptor(originalDescriptor) to retrieve the real descriptor specified at runtime by the user. If no descriptor was provided we should report an error. But the library tries to create a YamlContextualInput and waits for the actual descriptor being passed in beginStructure method call instead. Something like this:

descriptor.kind is SerialKind.CONTEXTUAL -> YamlInput.createFor(node, yaml, context, configuration, context.getContextualOrThrow(descriptor))

private fun SerializersModule.getContextualOrThrow(
    descriptor: SerialDescriptor
): SerialDescriptor = getContextualDescriptor(descriptor) ?: error("contextual serializer for type ${descriptor.capturedKClass} was not found")

Also, I have found some strange usage of SerialKind.CONTEXTUAL in tests. For example, ContextualSerializer in YamlReadingTest file. I am not sure this is the intended usage of this SerialKind type. I could not find anything like this in kotlinx.serialization README file. @charleskorn Could you please advise what was the intended behavior for this serializer? Probably, I just missing something here. I think you wanted to check something similar to @leinardi's case to see if the parser correctly identifies the type that might be different at runtime. Please, correct me if I am wrong here

charleskorn / kaml