Design a general JsonElement-like thing?

[ArcticLampyrid]

Currently the JsonElement is very specific to Json, however the similar feature can be used in other formats like Ctor, Yaml, MessagePack, etc.

What is your use-case and why do you need this feature? Similarly to JsonElement, sometimes we may need to process the complex struct manually. We may provide response for our api in multi formats (eg. Json for normal use & Ctor for better performance), then we need a more general way to handle xxxElement.

For a detailed example, all Discord APIs support both Json and ETF encodings.

ENCODING is the type of encoding for this connection to use. json and etf are supported. (Original)

Describe the solution you'd like The fundamental GeneralElement should be designed as RawMessage (inspired by Golang), which stores the values in a serializator-special binary format. This data may not have a clear TypeID, so we may not able to read or edit it without addtional information. But round-trip should be supported. RawMessage is compatible to Protobuf or other formats that do not have a clear runtime type info.

Some formats like Json or Ctor provides detailed type info to help us parse the data, then we can parse them into GeneralObjectElement, GeneralArrayElement or GeneralPrimitiveElement, which generalizd JsonObject, JsonArray and JsonPrimitive.

[altavir]

This one is similar: #222

[ArcticLampyrid]

222 is more likely to request a SAX-style reader but I’m asking a generic DOM.

[altavir]

It is about a generic tree-like object with string keys. I actually use exactly that in DataForge. But it is tailored for my own needs. I use it for DOM as well and it could be encoded to Json or XML.

[elizarov]

My 5 cents. All formats listed here: YAML, Cbor, MessagePack are based on JSON data information model, so using a JSONElement for them is totally appropriate. You cannot just call this thing GeneralElement, because it is NOT a general element. For example, XML Data Information Model is substantially different from the JSON data model and should use its own XMLElement with a different design, that would not fit well JSON-based formats.

[ArcticLampyrid]

@elizarov GeneralElement can be designed as RawMessage, just to provide a placeholder for round-trip. It's suitable for XML. GeneralObjectElement/GeneralArrayElement may be more json-special. But a very large part of formats have the concept of Map and Array, that's why it's tolerable to be designed generically.

What's more, the current architecture of kotlinx.serialization has had a preference to formats based on Map&List.

[altavir]

@elizarov the primary difference between JSON and XML is the treatment of same-name-siblings. In Json it is array, in XML, element order matters. I agree that the model is different, but in theory, it is possible to make some kind of common tooling to work with any tree-like structure.

[ArcticLampyrid]

For example, XML Data Information Model is substantially different from the JSON data model and should use its own XMLElement with a different design, that would not fit well JSON-based formats.

We can design a XmlNodeElement here as a subclass of GeneralElement for XML model. GeneralElement is at least useful for round-trip. And even we change the name GeneralElement to GeneralObjectModelElement, it's still useful in many cases.

[ArcticLampyrid]

GeneralElement can be designed as RawMessage, just to provide a placeholder for round-trip. It's suitable for XML.

In a concrete example:

data class Request(val identity: GeneralElement, val arg: String)
data class Response(val identity: GeneralElement, val result: String)

fun handleRequest(request: Request): Response {
    return Response(request.identity, "Hello, " + request.arg)
}

[aSemy]

As an incremental step it would be very helpful to break out the JsonElement classes and utility functions into a separate, independent, dependency.

This would help json5k https://github.com/xn32/json5k/issues/2 so it's possible to encode/decode to/from JsonElements, but does not require muddying the JSON5 code with KxS JSON serialization.

[sandwwraith]

@aSemy I doubt this is worth it or even possible, because JsonElements serializers heavily rely on functionality specific to Json format (see JsonEncoder and JsonDecoder)