Closed austinarbor closed 3 months ago
Can you not use Java reflection APIs to get the annotations from the class and maybe its fields? I don't quite see why Jackson needs to have an API for this when Java provides one out of the box.
@pjfanning if I am serializing Child
, how do I know if I am serializing it as a member of Parent
, or as its own independent entity?
@pjfanning if I am serializing
Child
, how do I know if I am serializing it as a member ofParent
, or as its own independent entity?
Serializers are usually registered for a particular class. For instance, you have class ChildSerializer : StdSerializer<Child>
- what other class is meant to be handled if you explicitly say you want to only serialize Child
instances.
Surely, Child has a reference to Parent. override fun serialize(obj: Child,
-- can't you check if obj is null or not null -- and then check if child.parent is null or not null?
my original example has the use case
data class Parent(val name: String) {
@JsonIgnoreProperties("parent")
var child: Child? = null
}
data class Child(
val name: String,
@JsonIgnoreProperties("child")
val parent: Parent,
)
My serializer will handle only serializing Child
in both cases. However, in the case of parent -> child
, child.parent
should not be serialized. In the case of child
, child.parent
should be. Whether child.parent
is null or not isn't a good enough indicator if it should be serialized..
val parent = Parent("parent")
val child = Child("child", parent)
parent.child = child
// if I serialize parent.child.parent i get infinite recursion
mapper.writeValueAsString(parent)
// if i serialize child.parent.child i get infinite recursion
mapper.writeValueAsString(child)
when serializing the parent, I should get
{ "name" : "parent", "child": { "name": "child" }}
and when serializing the child I should get
{ "name" : "child", "parent": { "name": "parent" }}
@austinarbor If infintie recursion is your problem, I think it is resolved in at least 2.17 version. Please refer below for reproducible test that runs and passes against Jackson Kotlin module version 2.17.
Note that serializing child outputs to "{\"name\":\"child\"}"
.
class `4501Test` {
data class Parent(val name: String) {
@JsonIgnoreProperties("parent")
var child: Child? = null
}
data class Child(
val name: String,
@JsonIgnoreProperties("child")
val parent: Parent,
)
class ChildSerializer : StdSerializer<Child>(Child::class.java) {
override fun serialize(obj: Child, gen: JsonGenerator, provider: SerializerProvider) {
gen.writeStartObject()
gen.writeFieldName("name")
gen.writeString(obj.name)
gen.writeEndObject()
}
}
@Test
fun `reproduce infinite recursion`() {
// Given
val simpleModule = SimpleModule().addSerializer(Child::class.java, ChildSerializer())
val mapper = jacksonObjectMapper()
.registerModules(simpleModule)
val parent = Parent("parent")
val child = Child("child", parent)
parent.child = child
// When & Then
assertEquals(
"{\"name\":\"parent\",\"child\":{\"name\":\"child\"}}",
mapper.writeValueAsString(parent)
)
assertEquals(
"{\"name\":\"child\"}",
mapper.writeValueAsString(child)
)
}
}
@JooHyukKim your ChildSerializer
is incorrect, it should be
class ChildSerializer : StdSerializer<Child>(Child::class.java) {
override fun serialize(obj: Child, gen: JsonGenerator, provider: SerializerProvider) {
gen.writeStartObject()
gen.writeFieldName("name")
gen.writeString(obj.name)
gen.writeFieldName("parent")
gen.writeObject(obj.parent)
gen.writeEndObject()
}
}
Running that, you get
Document nesting depth (1002) exceeds the maximum allowed (1000, from StreamWriteConstraints.getMaxNestingDepth()`) (through reference chain: ...
@austinarbor you can implement a ContextualSerializer to access annotations on the property
What @yawkat said -- Jackson design is such that ALL annotation access must come from "static" part of processing, when constructing and initializing (de)serializers and other handlers: this information is not (and will not) ever be passed during more dynamic serialization/deserialization code path.
Serializers and deserializers can however (and some do) store necessary information during construction/initialization, and ContextSerializer
s createContextual()
callback is where information is passed.
So information is not and will not be passed through serialize()
method. This is by design.
@yawkat @cowtowncoder thanks! ContextualSerializer seems to be working well - I did have a little bit of trouble figuring out the best way to "propagate" the annotations to downstream serializers. For example, I am working with Jetbrains' exposed ORM which has a LazySizedCollection
(their implementation of a list). I was trying to replicate the behavior of how the @JsonIgnoreProperties
annotation gets applied to a list of objects, for example
class MyObject {
@JsonIgnoreProperties("someProperty") // the annotation is applied to each item in the list
val others : List<OtherObject> = listOf()
}
I wasn't really able to replicate that functionality fully, is there a better approach to use than provider.findValueSerializer
below?
class LazySizedCollectionSerializer(private val bean: BeanProperty? = null) :
StdSerializer<LazySizedCollection<*>>(
LazySizedCollection::class.java,
),
ContextualSerializer {
override fun serialize(
value: LazySizedCollection<*>,
gen: JsonGenerator,
provider: SerializerProvider,
) {
gen.writeStartArray()
// if the value is loaded from the database already, serialize it
// otherwise just write an empty array
if (value.isLoaded()) {
for (item in value) {
if (item == null) {
gen.writeNull()
} else {
// for some reason, the calls to _other_ createContextual custom serializers
// all were passing a null bean property, even with this below
val ser = provider.findValueSerializer(item::class.java, bean)
ser.serialize(item, gen, provider)
}
}
}
gen.writeEndArray()
}
override fun createContextual(
prov: SerializerProvider?,
property: BeanProperty?,
): JsonSerializer<*> {
// the bean property here contains the JsonIgnoreProperties annotation information
return LazySizedCollectionSerializer(property)
}
}
Hmmh. Yeah, that is challenging thing to do; given that handling of elements (applying annotations to collection elements), it might make sense to instead use BeanSerializerModifier
.
And implementing custom List serializer it might make sense to extend a more advanced base class than StdSerializer
: f.ex
src/main/java/com/fasterxml/jackson/databind/ser/impl/IndexedListSerializer.java
extends AsArraySerializerBase
.
Hopefully some combination works.
The BeanSerializerModifier
idea actually helped a lot because it gave me access to the BeanProperties from the original bean analysis. I think I was able to get something working, does it look like I'm doing anything horribly wrong?
class SerializerModifier : BeanSerializerModifier() {
override fun modifySerializer(
config: SerializationConfig,
beanDesc: BeanDescription,
serializer: JsonSerializer<*>,
): JsonSerializer<*> {
if (beanDesc.beanClass.kotlin.isSubclassOf(Entity::class)) {
return EntitySerializer(serializer as JsonSerializer<Any>)
}
return serializer
}
}
class EntitySerializer(private val ser: JsonSerializer<Any>, private val bean: BeanProperty? = null)
: StdSerializer<Entity<*>>(Entity::class.java), ContextualSerializer {
private val ignoredProperties: Set<String>
init {
val ignored = mutableSetOf<String>()
if (bean != null) {
// the javadoc says to use the AnnotationInspector, but the ignored properties never change so
// i thought it would be more efficient to calculate them once up front
bean.getAnnotation(JsonIgnoreProperties::class.java)?.value?.forEach { ignored.add(it) }
bean.getContextAnnotation(JsonIgnoreProperties::class.java)?.value?.forEach {
ignored.add(it)
}
}
ignoredProperties = ignored
}
override fun serialize(entity: Entity<*>, gen: JsonGenerator, provider: SerializerProvider) {
gen.writeStartObject()
// right now this code looks like it could be replaced by the default serializer,
// but i am excluding some custom logic for brevity. i just want to demonstrate how
// i am determining the ignored properties
ser.properties()
.asSequence()
.filterNot { it.name in ignoredProperties }
.forEach { prop ->
gen.writeFieldName(prop.name)
val v = prop.member.getValue(entity)
if (v == null) {
gen.writeNull()
} else {
val valueSerializer = provider.findValueSerializer(v::class.java, prop)
valueSerializer.serialize(v, gen, provider)
}
}
gen.writeEndObject()
}
override fun createContextual(prov: SerializerProvider, property: BeanProperty?): JsonSerializer<*> {
return EntitySerializer(this.ser, property)
}
}
class LazySizedCollectionSerializer(private val bean: BeanProperty? = null) :
StdSerializer<LazySizedCollection<*>>(LazySizedCollection::class.java), ContextualSerializer {
override fun serialize(
value: LazySizedCollection<*>,
gen: JsonGenerator,
provider: SerializerProvider,
) {
if (value.isLoaded()) {
val ser = provider.findValueSerializer(List::class.java, bean)
ser.serialize(value.wrapper, gen, provider)
} else {
// value was not specified to be eagerly fetched from the database so write null
gen.writeNull()
}
}
override fun createContextual(provider: SerializerProvider?, property: BeanProperty?): JsonSerializer<*> {
return LazySizedCollectionSerializer(property)
}
}
That looks fine overall, except this:
bean.getContextAnnotation(JsonIgnoreProperties::class.java)
is probably not what you want: it'd be for class annotations of class that has property with List
value. While such defaulting is done for other uses, it seems incorrect here.
Actually... another thing worth considering is that BeanDescription
has lots of useful accessors to use... and I'd thing getIgnoredPropertyNames()
is what you'd want? It is based on both Annotations and Config overrides. So if that works, it's preferable over explicit annotation access.
And from performance perspective, having to dynamically call findValueSerializer()
for serialize()
can be quite inefficient. Not sure if that can be avoided, although default BeanSerializer
tries resolving these during createContextual()
(or maybe resolve()
), and even in the case of having to dynamically resolve serializers does lazy-caching using PropertySerializerMap
. But that gets rather complicated... but if you could reuse standard value serializers, that'd be supported out of the box.
Oh, one more thing: JsonSerializer
has "mutant factory method":
public JsonSerializer<?> withIgnoredProperties(Set<String> ignoredProperties)
so you can apply ignorals (serializers for which these do not apply, like scalar serializers, simply do return this
without error).
Anyway, I hope some of above is useful. Good progress so far! :)
Hmm... getIgnoredPropertyNames
seems to always be empty in the BeanDescription
, not sure why it's missing the values
Agreed this probably isn't the most performant - I opened a ticket on the Exposed YouTrack to add a feature which would let me optimize a bit more
The only reason I need the custom serializer is because I need to wrap the getValue call in a try/catch and ignore a specific exception and write null when it happens.
Exposed currently doesn't have a way to checking if a 1:1 association has been fetched yet, and if you try and fetch the association after the transaction is closed you get an exception. It's not pretty, but at the moment it's the only way to get it done (that I know of). If there's an easier way to implement writing a null
when a specific exception occurs on the getter invocation I would love to hear it
ser.properties()
.asSequence()
.filterNot { it.name in ignoredProperties }
.forEach { prop ->
gen.writeFieldName(prop.name)
try {
val v = prop.member.getValue(entity)
if (v == null) {
gen.writeNull()
} else {
val valueSerializer = provider.findValueSerializer(v::class.java, prop)
valueSerializer.serialize(v, gen, provider)
}
} catch(e: IllegalArgumentException) {
// if we get an IllegalArgumentException here with a certain underlying cause, it means
// the entity's association was not fetched and we want to just write null to the json
gen.writeNull()
}
}
I wonder...can I provide a custom Introspector for one class only that provides BeanProperty with a wrapped getValue
implementation? I could put the try/catch in the custom implementation and avoid the custom serializer entirely
There is no way to change AnnotationIntrospector
configured for an ObjectMapper
after construction, or to override it in any way.
Odd that BeanDescription
has empty/missing one; would be good to know if a test could be isolated to show the problem (without all the other surrounding code).
But in short term it is what it is -- depending, it could be that annotation only visible to List
serializer and you'd need it for element within.
Not quite sure what could be done here; closing for now. May be re-filed specific ask (and ref to this issue as background if that makes sense).
Is your feature request related to a problem? Please describe.
When writing a custom serializer, I'd like to be able to know the Jackson annotations that were on the entity being serialized. Serializing fields may be be intensive or cause recursion, so if the caller has specified to ignore that field, I would like to know before I try and serialize it
Describe the solution you'd like
Either
JsonGenerator
orSerializerProvider
(or some other static method) should provide a lookup mechanism to get the Jackson annotations related to the current object being serializedUsage example
Additional context
No response