Closed sedovalx closed 5 years ago
This can be done via using JsonElement
, see #175 and #276.
This functionality has been implemented and will be shipped with next release.
@sandwwraith Can you possibly share an example code to deal with Map<String, Any>
? 🙇
@hotchemi You can replace Map
type with JsonObject
@sandwwraith this isnt really a solution for orgs that already have maps heavily ingrained, which is more likely than having JsonObject everywhere. There should be some support for Map<String, Any> without converting it to a JsonObject or converting every map to <String, JsonElement>.
JsonObject
implements Map<String, JsonElement>
. I believe that Any
is not very type-safe and much more cumbersome to handle, but this JsonElement
itself can be easily converted to Any recursively.
Its about converting Any to JsonElement, not the other way around. Any isnt very type-safe but provides a big use case with Maps. As with other serializers it would just thrown an JsonException on unsupported types.
@dri94 did you find a way to convert Any
to JsonElement
in order to convert a Map<String, Any>
to JsonObject
?
It appears there is no way to serialize a dashed variable in Kotlin JS, JSONObject only works on JVM: https://github.com/JetBrains/kotlin-wrappers/issues/339
ERROR: "name contains illegal identifiers that can't appear in javascript identifiers"
styleManager = jsObject{
clearProperties = true
}
plugins = arrayOf(
"grapesjs-lory-slider",
"grapesjs-tabs",
"grapesjs-custom-code",
"grapesjs-touch",
"grapesjs-parser-postcss",
"grapesjs-tooltip",
"grapesjs-tui-image-editor",
"grapesjs-typed",
"grapesjs-style-bg",
"grapesjs-preset-webpage"
//"grapesjs-plugin-filestack"
)
pluginsOpts = jsObject<dynamic> {
`grapesjs-lory-slider` = jsObject<dynamic> {
sliderBlock = jsObject<dynamic> {
category = "Extra"
}
}
}
EDIT/SOLVED: this[ "grapesjs-lory-slider"]
This is how I am doing Map to JsonElement for now. I take all primitive to be string (In my case it is fine).
fun List<*>.toJsonElement(): JsonElement {
val list: MutableList<JsonElement> = mutableListOf()
this.forEach {
val value = it as? Any ?: return@forEach
when(value) {
is Map<*, *> -> list.add((value).toJsonElement())
is List<*> -> list.add(value.toJsonElement())
else -> list.add(JsonPrimitive(value.toString()))
}
}
return JsonArray(list)
}
fun Map<*, *>.toJsonElement(): JsonElement {
val map: MutableMap<String, JsonElement> = mutableMapOf()
this.forEach {
val key = it.key as? String ?: return@forEach
val value = it.value ?: return@forEach
when(value) {
is Map<*, *> -> map[key] = (value).toJsonElement()
is List<*> -> map[key] = value.toJsonElement()
else -> map[key] = JsonPrimitive(value.toString())
}
}
return JsonObject(map)
}
I was expecting to be able to use Map<String, Serializable?>
instead to get the proper serializer, but looks like this not working either.
No way to get a generic interface for all the possible serializable types?
Well to keep this more generic, I'm using a Map where all the elements have a serializer and we can use the kotlin reflection to get the right one for the handled type, so we can get an object out of the map with just:
fun buildJsonObject(other: Map<String, Any?>) : JsonElement {
val jsonEncoder = Json{ encodeDefaults = true } // Set this accordingly to your needs
val map = emptyMap<String, JsonElement>().toMutableMap()
other.forEach {
map[it.key] = if (it.value != null)
jsonEncoder.encodeToJsonElement(serializer(it.value!!::class.starProjectedType), it.value)
else JsonNull
}
return JsonObject(map)
}
And this will still throw a SerializationException
in case the value type has not a serializer available.
However, I'm not still fully happy as I'd prefer some more generic serializable type so that can be used with binary when using CBOR serialization, and so where the encoding happens only at the moment we call the Json.encodeToString
or Cbor.encodeToByteArray
depending whether the serializer supports or not the binary format.
Not to mention that a such built object would just fail with Cbor (Got an error while parsing: java.lang.IllegalStateException: This serializer can be used only with Json format.Expected Encoder to be JsonEncoder, got class kotlinx.serialization.cbor.internal.CborMapWriter
).
So, to handle the generic serializer case (such as binary ones), I've crafted some raw KSerializer
for Any?
that manually serializes the dynamic type of the element other than the value itself.
So basically mimicking what Polymorphic does, I'm not using any experimental or internal APIs but some of them could improve the result, like reusing the type serialName if any (even though, I'm not sure how i can deserialize that).
In the JSON case it could be probably optimized removing the type at all when using a Primitive one. (EDIT: this is done now)
Here's a gist, but suggestions are welcome: https://gist.github.com/3v1n0/ecbc5e825e2921bd0022611d7046690b
See https://youtrack.jetbrains.com/issue/KTOR-3063, also thanks to @kabirsaheb for the first draft of toJsonElement(). The linked youtrack issue has a more advanced mitigation strategy for this issue, where serializing Map, List, String, Number, Boolean, Enum and null is supported.
Based on @kabirsaheb, I use the next:
fun Any?.toJsonElement(): JsonElement =
when (this) {
null -> JsonNull
is Map<*, *> -> toJsonElement()
is Collection<*> -> toJsonElement()
is Boolean -> JsonPrimitive(this)
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
is Enum<*> -> JsonPrimitive(this.toString())
else -> throw IllegalStateException("Can't serialize unknown type: $this")
}
private fun Collection<*>.toJsonElement(): JsonElement {
val list: MutableList<JsonElement> = mutableListOf()
this.forEach { value ->
when (value) {
null -> list.add(JsonNull)
is Map<*, *> -> list.add(value.toJsonElement())
is Collection<*> -> list.add(value.toJsonElement())
is Boolean -> list.add(JsonPrimitive(value))
is Number -> list.add(JsonPrimitive(value))
is String -> list.add(JsonPrimitive(value))
is Enum<*> -> list.add(JsonPrimitive(value.toString()))
else -> throw IllegalStateException("Can't serialize unknown collection type: $value")
}
}
return JsonArray(list)
}
private fun Map<*, *>.toJsonElement(): JsonElement {
val map: MutableMap<String, JsonElement> = mutableMapOf()
this.forEach { (key, value) ->
key as String
when (value) {
null -> map[key] = JsonNull
is Map<*, *> -> map[key] = value.toJsonElement()
is Collection<*> -> map[key] = value.toJsonElement()
is Boolean -> map[key] = JsonPrimitive(value)
is Number -> map[key] = JsonPrimitive(value)
is String -> map[key] = JsonPrimitive(value)
is Enum<*> -> map[key] = JsonPrimitive(value.toString())
else -> throw IllegalStateException("Can't serialize unknown type: $value")
}
}
return JsonObject(map)
}
like is code
@kotlinx.serialization.Serializable
data class MyDataClass(var s: String)
println(mapOf("a" to 1, "b" to "2", "c" to listOf(MyDataClass("3"))).toJsonElement2())
will be throw IllegalStateException
I'm surprised that such conversion is not supported by the library, it seems to be quite common when working with Java code. Thanks for the workarounds though!
package test2
import kotlinx.serialization.*
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.json.*
import test1.printlnIt
import java.time.OffsetDateTime
import java.time.format.DateTimeFormatter
import kotlin.test.Test
@Serializable
data class AnyData(
val name: String,
@Serializable(with = AnyValueSerializer::class)
val anyValue: Any?,
val anyList: List<@Serializable(with = AnyValueSerializer::class) Any?>,
val anyMap: Map<@Serializable(with = AnyValueSerializer::class) Any?, @Serializable(with = AnySerializer::class) Any?>,
@Serializable(with = DataFrameSerializer::class)
val dataFrame: Map<String,List<@Contextual Any?>>, // {date:List<OffsetDateTime>, price:List<Double>}
)
private fun Any?.toJsonPrimitive(): JsonPrimitive {
return when (this) {
null -> JsonNull
is JsonPrimitive -> this
is Boolean -> JsonPrimitive(this)
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
// add custom convert
else -> throw Exception("不支持类型:${this::class}")
}
}
private fun JsonPrimitive.toAnyValue():Any?{
val content = this.content
if (this.isString){
// add custom string convert
return content
}
if (content.equals("null", ignoreCase = true)){
return null
}
if (content.equals("true", ignoreCase = true)){
return true
}
if (content.equals("false", ignoreCase = true)){
return false
}
val intValue = content.toIntOrNull()
if (intValue!=null){
return intValue
}
val longValue = content.toLongOrNull()
if (longValue!=null){
return longValue
}
val doubleValue = content.toDoubleOrNull()
if (doubleValue!=null){
return doubleValue
}
throw Exception("未知值:${content}")
}
object AnyValueSerializer : KSerializer<Any?> {
private val delegateSerializer = JsonPrimitive.serializer()
override val descriptor = delegateSerializer.descriptor
override fun serialize(encoder: Encoder, value: Any?) {
encoder.encodeSerializableValue(delegateSerializer, value.toJsonPrimitive())
}
override fun deserialize(decoder: Decoder): Any? {
val jsonPrimitive = decoder.decodeSerializableValue(delegateSerializer)
return jsonPrimitive.toAnyValue()
}
}
/**
* Convert Any? to JsonElement
*/
private fun Any?.toJsonElement(): JsonElement{
return when (this) {
null -> JsonNull
is JsonElement -> this
is Boolean -> JsonPrimitive(this)
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
is Iterable<*> -> JsonArray(this.map { it.toJsonElement() })
// !!! key simply converted to string
is Map<*, *> -> JsonObject(this.map { it.key.toString() to it.value.toJsonElement() }.toMap())
// add custom convert
else -> throw Exception("不支持类型 ${this::class}=${this}}")
}
}
private fun JsonElement.toAnyOrNull():Any?{
return when (this) {
is JsonNull -> null
is JsonPrimitive -> toAnyValue()
// !!! key convert back custom object
is JsonObject -> this.map { it.key to it.value.toAnyOrNull() }.toMap()
is JsonArray -> this.map { it.toAnyOrNull() }
}
}
object AnySerializer : KSerializer<Any?> {
private val delegateSerializer = JsonElement.serializer()
override val descriptor = delegateSerializer.descriptor
override fun serialize(encoder: Encoder, value: Any?) {
encoder.encodeSerializableValue(delegateSerializer, value.toJsonElement())
}
override fun deserialize(decoder: Decoder): Any? {
val jsonPrimitive = decoder.decodeSerializableValue(delegateSerializer)
return jsonPrimitive.toAnyOrNull()
}
}
object DataFrameSerializer : KSerializer<Map<String,List<Any?>>> {
private val stdDateTimeFormatter: DateTimeFormatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ssZ")
private val delegateSerializer = JsonObject.serializer()
override val descriptor = delegateSerializer.descriptor
override fun serialize(encoder: Encoder, value: Map<String,List<Any?>>) {
val jsonObject = JsonObject(value.mapValues {
when(it.key){
"date" -> JsonArray(it.value.map { v ->
JsonPrimitive(stdDateTimeFormatter.format(v as OffsetDateTime))
})
"price" -> JsonArray(it.value.map { v ->
JsonPrimitive(v as Double)
})
else -> throw IllegalStateException("Unknown key${it.key}")
}
})
encoder.encodeSerializableValue(delegateSerializer, jsonObject)
}
override fun deserialize(decoder: Decoder): Map<String,List<Any?>> {
val jsonObject = decoder.decodeSerializableValue(delegateSerializer)
val map = jsonObject.mapValues {
when (it.key) {
"date" -> it.value.jsonArray.map { v ->
OffsetDateTime.parse(v.jsonPrimitive.content, stdDateTimeFormatter)
}
"price" -> it.value.jsonArray.map { v -> v.jsonPrimitive.double }
else -> throw IllegalStateException("Unknown key${it.key}")
}
}
return map
}
}
class SerialTest3 {
@Test
fun test1() {
val anyData = AnyData(
name = "hello",
anyValue = null,
anyList = listOf("hi", 123, Long.MAX_VALUE, 2.7, true, null),
anyMap = mapOf(
true to listOf(123, 25.0, false),
false to listOf(1.2, 1.3, 0.5),
"RussianDoll" to mapOf(true to listOf(123, 25.0, false), false to listOf(1.2, 1.3, 0.5))
),
dataFrame = mapOf(
"date" to listOf(OffsetDateTime.now(), OffsetDateTime.now().plusDays(1), OffsetDateTime.now().plusDays(2)),
"price" to listOf(1.2, 1.3, 0.5)
),
)
val json = Json.encodeToString(anyData)
println("json: $json")
println("class: ${Json.decodeFromString<AnyData>(json)}")
}
}
output:
json: {"name":"hello","anyValue":null,"anyList":["hi",123,9223372036854775807,2.7,true,null],"anyMap":{"true":[123,25.0,false],"false":[1.2,1.3,0.5],"RussianDoll":{"true":[123,25.0,false],"false":[1.2,1.3,0.5]}},"dataFrame":{"date":["2022-04-30 11:42:06+0800","2022-05-01 11:42:06+0800","2022-05-02 11:42:06+0800"],"price":[1.2,1.3,0.5]}}
class: AnyData(name=hello, anyValue=null, anyList=[hi, 123, 9223372036854775807, 2.7, true, null], anyMap={true=[123, 25.0, false], false=[1.2, 1.3, 0.5], RussianDoll={true=[123, 25.0, false], false=[1.2, 1.3, 0.5]}}, dataFrame={date=[2022-04-30T11:42:06+08:00, 2022-05-01T11:42:06+08:00, 2022-05-02T11:42:06+08:00], price=[1.2, 1.3, 0.5]})
Thanks, @kabirsaheb for your solution. Here another approach based on your solution and the @migueltorcha one:
fun Collection<*>.toJsonElement(): JsonElement = JsonArray(mapNotNull { it.toJsonElement() })
fun Map<*, *>.toJsonElement(): JsonElement = JsonObject(
mapNotNull {
(it.key as? String ?: return@mapNotNull null) to it.value.toJsonElement()
}.toMap(),
)
fun Any?.toJsonElement(): JsonElement = when (this) {
null -> JsonNull
is Map<*, *> -> toJsonElement()
is Collection<*> -> toJsonElement()
else -> JsonPrimitive(toString())
}
fun Collection<*>.toJsonElement(): JsonElement = JsonArray(mapNotNull { it.toJsonElement() }) fun Map<*, *>.toJsonElement(): JsonElement = JsonObject( mapNotNull { (it.key as? String ?: return@mapNotNull null) to it.value.toJsonElement() }.toMap(), ) fun Any?.toJsonElement(): JsonElement = when (this) { null -> JsonNull is Map<*, *> -> toJsonElement() is Collection<*> -> toJsonElement() else -> JsonPrimitive(toString()) }
Hi, do I need to configure anything else to make this work in ktor js client?
I have this error:
'StandaloneCoroutine is cancelling', Caused by: 'Fail to prepare request body for sending.
The body type is: class JsonObject, with Content-Type: null.
If you expect serialized body, please check that you have installed the corresponding plugin(like `ContentNegotiation`) and set `Content-Type` header.'
Here is my configuration:
val httpClient = HttpClient {
expectSuccess = true
install(ContentNegotiation) {
json()
}
}
Ignore my comment, I need to set manually contentType(ContentType.Application.Json)
when creating the request.
I would expect the content negotiation plugin do it automatically.
try this:
@OptIn(InternalSerializationApi::class)
fun Any?.toJsonElement(): JsonElement =
when (this) {
null -> JsonNull
is Map<*, *> -> toJsonElement()
is Collection<*> -> toJsonElement()
is Boolean -> JsonPrimitive(this)
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
is Enum<*> -> JsonPrimitive(this.toString())
else -> this.javaClass.kotlin.serializer().let { json.encodeToJsonElement(it, this) }
}
private fun Collection<*>.toJsonElement(): JsonElement =
JsonArray(this.map { it.toJsonElement() })
private fun Map<String, Any?>.toJsonElement(): JsonElement {
return JsonObject(this.mapValues { it.value.toJsonElement() })
}
Base on @migueltorcha
considering Array<*>
and XxxArray
.
fun Any?.toJsonElement(): JsonElement = when(this) {
null -> JsonNull
is Map<*, *> -> toJsonElement()
is Collection<*> -> toJsonElement()
is ByteArray -> this.toList().toJsonElement()
is CharArray -> this.toList().toJsonElement()
is ShortArray -> this.toList().toJsonElement()
is IntArray -> this.toList().toJsonElement()
is LongArray -> this.toList().toJsonElement()
is FloatArray -> this.toList().toJsonElement()
is DoubleArray -> this.toList().toJsonElement()
is BooleanArray -> this.toList().toJsonElement()
is Array<*> -> toJsonElement()
is Boolean -> JsonPrimitive(this)
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
is Enum<*> -> JsonPrimitive(this.toString())
else -> {
throw IllegalStateException("Can't serialize unknown type: $this")
}
}
fun Map<*, *>.toJsonElement(): JsonElement {
val map = mutableMapOf<String, JsonElement>()
this.forEach {key, value ->
key as String
map[key] = value.toJsonElement()
}
return JsonObject(map)
}
fun Collection<*>.toJsonElement(): JsonElement {
return JsonArray(this.map { it.toJsonElement() })
}
fun Array<*>.toJsonElement(): JsonElement {
return JsonArray(this.map { it.toJsonElement() })
}
fun main(args: Array<String>) {
val obj = mapOf<String, Any>(
"int" to 1,
"bool" to true,
"float" to 1.2f,
"double" to 1.2,
"arrayInt" to intArrayOf(1, 2,3),
"arrayInt2" to arrayOf(1, 2,3),
"arrayString" to arrayOf("foo", "bar"),
"listDouble" to listOf(1.1, 2.2, 3.3),
"listString" to listOf("goo", "baz"),
"mapInt" to mapOf("1" to 1, "2" to 2),
).toJsonElement()
println(Json.encodeToString(obj))
}
{
"int":1,
"bool":true,
"float":1.2,
"double":1.2,
"arrayInt":[
1,
2,
3
],
"arrayInt2":[
1,
2,
3
],
"arrayString":[
"foo",
"bar"
],
"listDouble":[
1.1,
2.2,
3.3
],
"listString":[
"goo",
"baz"
],
"mapInt":{
"1":1,
"2":2
}
}
Using some of what has been given in this issue as well as messing around with it myself, here is some code that (should) be able to serialize anything that kotlinx.serialization can serialize using Json.encodeToJsonElement
:
Do note: @file:OptIn(ExperimentalUnsignedTypes::class, ExperimentalSerializationApi::class)
needs to be added at the top.
If you wish to remove this, just edit the code and remove all references to any unsigned types.
internal fun Any?.toJsonElement(): JsonElement {
val serializer = this?.let { Json.serializersModule.serializerOrNull(this::class.java) }
return when {
this == null -> JsonNull
serializer != null -> Json.encodeToJsonElement(serializer, this)
this is Map<*, *> -> toJsonElement()
this is Array<*> -> toJsonElement()
this is BooleanArray -> toJsonElement()
this is ByteArray -> toJsonElement()
this is CharArray -> toJsonElement()
this is ShortArray -> toJsonElement()
this is IntArray -> toJsonElement()
this is LongArray -> toJsonElement()
this is FloatArray -> toJsonElement()
this is DoubleArray -> toJsonElement()
this is UByteArray -> toJsonElement()
this is UShortArray -> toJsonElement()
this is UIntArray -> toJsonElement()
this is ULongArray -> toJsonElement()
this is Collection<*> -> toJsonElement()
this is Boolean -> JsonPrimitive(this)
this is Number -> JsonPrimitive(this)
this is String -> JsonPrimitive(this)
this is Enum<*> -> JsonPrimitive(this.name)
this is Pair<*, *> -> this.toList().toJsonElement()
this is Triple<*, *, *> -> this.toList().toJsonElement()
else -> error("Can't serialize '$this' as it is of an unknown type")
}
}
internal fun Map<*, *>.toJsonElement(): JsonElement {
return buildJsonObject {
forEach { (key, value) ->
if (key !is String)
error("Only string keys are supported for maps")
put(key, value.toJsonElement())
}
}
}
internal fun Collection<*>.toJsonElement(): JsonElement = buildJsonArray {
forEach { element ->
add(element.toJsonElement())
}
}
internal fun Array<*>.toJsonElement(): JsonElement = buildJsonArray {
forEach { element ->
add(element.toJsonElement())
}
}
internal fun BooleanArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun ByteArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun CharArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it.code)) } }
internal fun ShortArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun IntArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun LongArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun FloatArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun DoubleArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun UByteArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun UShortArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun UIntArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
internal fun ULongArray.toJsonElement(): JsonElement = buildJsonArray { forEach { add(JsonPrimitive(it)) } }
I'm trying to figure out a way to serialize an arbitrary
Map<String, Any>
into JSON. In my case, I can guarantee that in runtime any value of the map is either a primitive, a list or a map. In case of lists and maps, the values of them are either primitives or lists or maps of the same pattern. Seems, it lays nicely on a json object.I do not need to deserialize such an object, serialization only.
For now, I'm trying to write a custom serializer for such case but no success yet. How can it be done for both JVM and JS?