[BUG] Config option `ignoreUnknownKeys` breaks when there are unknown keys at the end

[RaphaelTarita]

I was trying to parse level.dat to get the level name, but it turns out that it doesn't work with the models

@OptIn(ExperimentalNbtApi::class)
@NbtRoot(name = "")
@Serializable
data class LevelRoot(
    @SerialName("Data")
    val data: LevelData
)

@SerialName("Data")
@Serializable
data class LevelData(
    @SerialName("LevelName")
    val levelName: String
)

and the config

Nbt {
    variant = NbtVariant.Java
    compression = NbtCompression.Gzip
    ignoreUnknownKeys = true
}

The following exception + stacktrace is thrown:

Exception in thread "main" java.lang.IllegalStateException: Unexpected TAG_End
    at net.benwoodworth.knbt.internal.NbtReaderKt.discardTag(NbtReader.kt:203)
    at net.benwoodworth.knbt.internal.ClassNbtDecoder.handleUnknownKey(NbtDecoder.kt:253)
    at net.benwoodworth.knbt.internal.ClassNbtDecoder.decodeElementIndex(NbtDecoder.kt:269)
    at msw.server.core.model.world.LevelData$$serializer.deserialize(leveldat.kt:17)
    at msw.server.core.model.world.LevelData$$serializer.deserialize(leveldat.kt:17)
    at kotlinx.serialization.encoding.Decoder$DefaultImpls.decodeSerializableValue(Decoding.kt:260)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:16)
    at net.benwoodworth.knbt.AbstractNbtDecoder.decodeSerializableValue(NbtDecoding.kt:84)
    at net.benwoodworth.knbt.internal.BaseNbtDecoder.decodeSerializableValue(NbtDecoder.kt:177)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:43)
    at net.benwoodworth.knbt.internal.BaseNbtDecoder.decodeSerializableValue(NbtDecoder.kt:180)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableElement(AbstractDecoder.kt:70)
    at msw.server.core.model.world.LevelRoot$$serializer.deserialize(leveldat.kt:8)
    at msw.server.core.model.world.LevelRoot$$serializer.deserialize(leveldat.kt:8)
    at kotlinx.serialization.encoding.Decoder$DefaultImpls.decodeSerializableValue(Decoding.kt:260)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:16)
    at net.benwoodworth.knbt.AbstractNbtDecoder.decodeSerializableValue(NbtDecoding.kt:84)
    at net.benwoodworth.knbt.internal.BaseNbtDecoder.decodeSerializableValue(NbtDecoder.kt:177)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:43)
    at net.benwoodworth.knbt.internal.BaseNbtDecoder.decodeSerializableValue(NbtDecoder.kt:180)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableElement(AbstractDecoder.kt:70)
    at kotlinx.serialization.encoding.CompositeDecoder$DefaultImpls.decodeSerializableElement$default(Decoding.kt:535)
    at net.benwoodworth.knbt.NbtRootDeserializer.deserialize(NbtRoot.kt:66)
    at kotlinx.serialization.encoding.Decoder$DefaultImpls.decodeSerializableValue(Decoding.kt:260)
    at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:16)
    at net.benwoodworth.knbt.AbstractNbtDecoder.decodeSerializableValue(NbtDecoding.kt:84)
    at net.benwoodworth.knbt.internal.BaseNbtDecoder.decodeSerializableValue(NbtDecoder.kt:177)
    at net.benwoodworth.knbt.NbtFormatKt.decodeFromNbtReader(NbtFormat.kt:67)
    at net.benwoodworth.knbt.Nbt.decodeFromSource(Nbt.kt:41)
    at net.benwoodworth.knbt.JvmStreamsKt.decodeFromStream(JvmStreams.kt:35)
    at [invocation of Nbt.decodeFromStream<LevelRoot>(inputStream)]

I found out that this exception is thrown when

1. ignoreUnknownKeys is true
2. the model classes don't cover all keys present in the nbt file
3. the last key of a compound is unknown

I started with testing this with level.dat, trying to access [root]/Data/LevelName. That didn't work because there are other keys after LevelName that are not present in the model. Then I tested it with a constructed nbt file that basically just contained

[root]: {
    Data: {
        LevelName: "test"
    }
}

This could be deserialized into the models. Then, I added a random key that is not present in the models:

[root]: {
    Data: {
        ScheduledEvents: []
        LevelName: "test"
    }
}

This still works, since ignoreUnknownKeys is turned on. However, when I swap the positions of ScheduledEvents and LevelName:

[root]: {
    Data: {
        LevelName: "test"
        ScheduledEvents: []
    }
}

The above error is thrown.

Kotlin 1.5.30 KXS 1.2.2 knbt 0.9.1

[BenWoodworth]

Thanks for reporting this! I'll take a look at what's going on...

NbtReader.kt:203

Reproduction:

import kotlinx.serialization.*
import net.benwoodworth.knbt.*

@NbtRoot(name = "")
@Serializable
data class LevelRoot(
    @SerialName("Data")
    val data: LevelData
)

@SerialName("Data")
@Serializable
data class LevelData(
    @SerialName("LevelName")
    val levelName: String
)

fun main() {
    val nbt = Nbt {
        variant = NbtVariant.Java
        compression = NbtCompression.Gzip
        ignoreUnknownKeys = true
    }

    val levelTag = buildNbtCompound("") {
        putNbtCompound("Data") {
            put("LevelName", "test")
            putNbtList<Nothing>("ScheduledEvents") {}
        }
    }

    nbt.decodeFromNbtTag<LevelRoot>(levelTag) // error
}

[BenWoodworth]

Could you give 0.9.2-SNAPSHOT a try?

You'll need to add the maven snapshot repo:
https://s01.oss.sonatype.org/content/repositories/snapshots/

[RaphaelTarita]

Just tried, works with 0.9.2-SNAPSHOT What I tried (and therefore validated): parsing multiple different level.dat files using the above mentioned model classes, then extracting [root]/Data/LevelName.

Thanks!

[BenWoodworth]

Awesome!! 0.9.2 should be published soon with the fix :)