Allow working with JSON5 without `@Serializable` classes

aSemy commented 1 year ago

I would like to be able to encode or decode JSON5 without using @Serializable classes.

Context

This is useful for tweaking JSON5 elements during serialization (for example, in a custom KSerializer), or for use in contexts where the Kotlinx Serialization plugin isn't available (Gradle config, .main.kts scripts)

KxS `JsonElement`

Kotlinx Serialization already provides JsonElement for this purpose.

Aside from direct conversions between strings and JSON objects, Kotlin serialization offers APIs that allow other ways of working with JSON in the code. For example, you might need to tweak the data before it can parse or otherwise work with such an unstructured data that it does not readily fit into the typesafe world of Kotlin serialization.

https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/json.md#json-elements

`JsonElement` example

import kotlinx.serialization.decodeFromString
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject

fun main() {
  val jsonObj: JsonObject = Json.decodeFromString("""
    {
      "some": "json",
      "count": 123
    }
  """.trimIndent())

  println(jsonObj)
}

Options

I think there are two options

json5k re-implements the JsonElement code, but for JSON5. This is mostly an exercise in renaming!
json5k re-uses the existing JsonElement code. Since JSON and JSON5 are so similar, I suspect this is the easiest approach.

aSemy commented 1 year ago

I've thought about this more and I think option 2 (re-using existing JsonElement) would require Json5k adds a dependency on kotlinx.serialization JSON - which is not ideal. I think option 1 would be best.

xn32 commented 1 year ago

I agree, this is definitely an important feature. I am still undecided about the two options, though.

On the one hand, a forced dependency on the JSON format will be undesired noise for most application scenarios of the library and should therefore be avoided. On the other hand, returning a JsonElement instance would allow clients of the library to reuse code from existing JsonTransformingSerializer implementations.

The ideal solution would probably be a format-independent version of JsonElement in the core that we can hook into. So instead of JsonArray in the JSON format, a ValueSequence class in the core would be exactly what we need.

Let me think about it while I work on the Multiplatform version! :slightly_smiling_face:

aSemy commented 1 year ago

There have been a few relevant updates from KxS JSON that I think are worth considering when implementing an equivalent in json5k. I'm mentioning them because they might help guide development in json5k.

JsonUnquotedLiteral() https://github.com/Kotlin/kotlinx.serialization/pull/2041 can be used to encode raw content. This can be used to encode types not supported in Kotlin Multiplatform, like BigDecimal and BigInteger, but also raw JSON content if necessary

This was done with a special 'marker' SerialDescriptor
Create JsonPrimitives for unsigned numbers https://github.com/Kotlin/kotlinx.serialization/pull/2160

Of particular note is the developer's comment saying (paraphrased) "adding nullable parameters for JsonPrimitive functions is a mistake."
Adding utility functions to the JsonObject/JsonArray for adding multiple elements

I actually think this is a bit messy and I think would be better implemented with additional MutableJsonObject and MutableJsonArray types, to complement the immutable versions.

context: https://github.com/Kotlin/kotlinx.serialization/issues/2308

Proposal

With the previous points in mind, I'd propose the following types:

sealed interface Json5Element

//region Json5 primitives
sealed interface Json5Primitive : Json5Element

/** internal primitive implementation, containing metadata about the content and how to encode/decode it */
internal class Json5Literal : Json5Primitive()

object Json5Null : Json5Primitive
//endregion

//region primitive constructors
fun Json5Primitive(value: String): Json5Primitive = // ...
fun Json5Primitive(value: Int): Json5Primitive = // ...
fun Json5Primitive(value: Boolean): Json5Primitive = // ...
// ...

fun Json5Primitive(value: Nothing?): Json5Null = Json5Null
//endregion

//region Json5 structures
interface Json5Object : Json5Element, Map<String, Json5Element>
interface MutableJson5Object : Json5Object, MutableMap<String, Json5Element>

interface Json5Array : Json5Element, List<Json5Element>
interface MutableJson5Array : Json5Array, MutableList<Json5Element>
//endregion

Notes

sealed interface vs class

I used sealed interface instead of sealed class, but both would probably work. I think using an interface allows the ABI to be much more controlled and minimal, but the interfaces do require implementation classes (e.g. internal class MutableJson5MapImpl : MutableJson5Map).

Serializers

I didn't include the @Serializable(with = ...) annotations, but they would be required

No nullable Json5Primitive parameters

The Json5Primitive function parameters aren't nullable, instead there's a Json5Primitive(value: Nothing?) function that will handle null https://github.com/Kotlin/kotlinx.serialization/pull/1907

Mutable type hierarchy

MutableJson5Object implements Json5Object, to match the List/MutableList pattern in Kotlin stdlib (and the same for Json5Array)

Raw encoding

Raw encoding is not critical for an initial implementation, but the function would look like this:

fun Json5UnquotedLiteral(value: String): Json5Element = // ...

Builder functions

Because there's a mutable variants of object and array, the builder functions would be simpler than KxS JSON.

fun buildJson5Object(build: MutableJson5Object.() -> Unit): Json5Object {
  // ...
}

fun buildJson5Array(build: MutableJson5Array.() -> Unit): Json5Array {
  // ...
}

Comments

Something that JSON doesn't have is comments. How should json5k handle them?

Perhaps this can be handled with a specific coerceToInlineType descriptor?

Or maybe comments need to be a specific subtype?

public interface Json5Comment : Json5Element

xn32 / json5k