Ignore fields during serialization that are set to null

[JohannRosenberg]

There needs to be a simple way to mark a field so that it will not be serialized when its value is set to null. This needs to be done on a field-by-field basis and not just on the class itself (although an additional way of marking the entire class would be great when you want to ignore any field in the class). Something like this:

@Serializable
class MyData {
  @ignorenull
   val somefield: Int?
}

[sandwwraith]

195 ? Alternatively, you can add null default value val somefield: Int? = null and it will be omitted automatically with json setting encodeDefaults = false

[jrivany]

I've been fighting with the same issue now, and so far the only solution has been writing several custom serializers.

I have a pretty standard use-case where we have several data classes that have non-null defaults on nullable values.

@Serializable
data class Config(
    val number: Double? = 1.0,
    // Assume several other similar fields...
)

The receiving end treats the payload as an update, if a field is omitted then it is not updated; otherwise the value is updated and persisted, even if it is null. I have no control over this end of the process.

So far I've written multiple serializers with Surrogate classes, inspired by this example that treat null as the default, so that I can use the @EncodeDefaults(EncodeDefaults.Mode.NEVER) on them.

In some cases I've been able to get away with a simpler JsonTransformingSerializer implementation.

I would highly appreciate any advice on reducing the amount of boiler plate here with the currently available library versions.

This is a lot of extra boiler plate, and is becoming increasingly tedious with dozens of such cases.

The basic idea of a field/property level annotation that could denote whether or not to encode a null value, seems pretty straightforward and simple, and has been asked for several times. I don't think that the explicitNulls configuation is sufficient for this use-case.

First of all it's global, and it could easily be something that is only needed on a case/field-by-field basis (as it is in my use-case). Second, it specifically doesn't suffice for the case where a value has a non-null default according to its documentation:

When this flag is disabled properties with `null` values without default are not encoded;
...

This is a pretty big pain-point for my current project. This also isn't just unique to my current project. I've worked with several other APIs in other projects that treat patch updates similarly.

Otherwise this is easily the best serialization library for kotlin available!

[jrivany]

I've managed to come up with a work-around that I think suits my purposes for now.

A slight secondary pain-point was being able to "wrap" the plugin generated Serializer globally, rather than when it appears as a field (since all the serialization in my application happens in a single inline function). Luckily, this is pretty easy to solve by dynamically resolving the top level serializer.

There are a number of optimizations that should be made, and it currently doesn't support nested structures, but here's a basic proof of concept:

@Target(AnnotationTarget.PROPERTY)
@Retention(AnnotationRetention.RUNTIME)
@SerialInfo
annotation class OmitOnNull

internal class NullableFilterSerializer<T: Any>(
        private val omitableFields: List<String>,
        serializer: KSerializer<T>
): JsonTransformingSerializer<T>(serializer) {
    override fun transformSerialize(element: JsonElement): JsonElement {
        if (element !is JsonObject) return element
        return JsonObject(element.toMutableMap().apply {
            omitableFields.forEach {
                if (element[it] == JsonNull) {
                    remove(it)
                }
            }
        })
    }
}

internal inline fun <reified T: Any> filterdSerializer(): KSerializer<T> {
    val serializer = serializer<T>()
    val fields = mutableListOf<String>()
    for (i in 0 until serializer.descriptor.elementsCount) {
        val field = serializer.descriptor.getElementName(i)
        if (serializer.descriptor.getElementAnnotations(i).find { it is OmitOnNull } != null) {
            fields.add(field)
        }
    }
    return if (fields.isNotEmpty()) {
        NullableFilterSerializer(fields, serializer)
    } else {
        serializer
    }
}

@Serializable
class Payload(
        val data: String,
        @OmitOnNull
        val other: String? = "",
)

// Usage:
Json.encodeToString(filterdSerializer(), Payload("hello", null))
// returns:
// {"data":"hello"}

[sandwwraith]

@jrivany I think it's a documentation a bit misleading, but explicitNulls = false should drop nulls regardless whether property has default value or not

[jrivany]

@sandwwraith Thanks, that's really helpful to know.

Unfortunately in my case there are also lots of places where given the same object graph, we'd need the setting to change on a per-field basis, so I still think having an annotation equivalent, much like the @EncodeDefault one, would be very useful.

[gm-vm]

I have a similar problem when using explicitNulls = false. I almost never want to encode or decode nulls, but there are still some cases in which I have to. This is generally needed when I need to perform some update: null tells the backend that it needs to delete the previous value, a missing property means that it needs to keep whatever is there already.

I know this is exactly the kind of problem that explicitNulls = true tackles, but unfortunately the risk of breaking things in unexpected ways is really high with legacy applications that assumed the behavior of explicitNulls = false for years, so I'd rather not use that.

What I've come up with is this:

@Serializable
@JvmInline
value class ExplicitNull<T>(val value: T?)

@Serializable
class Test(
    val property: ExplicitNull<Double>? = ExplicitNull(1.0),
)

fun main() {
    val json = Json {
        encodeDefaults = true
        explicitNulls = false
    }
    println(json.encodeToString(Test.serializer(), Test(null)))
    println(json.encodeToString(Test.serializer(), Test(ExplicitNull(null))))
    println(json.encodeToString(Test.serializer(), Test()))
}

The output of this is the following:

{}
{"property":null}
{"property":1.0}

The extra wrapper is maybe annoying, but it is good enough for me since this would be the exception rather than the norm.

But, I now wonder if this unintended and likely to break in future, especially considering that all of this is still experimental.