Open doljae opened 2 years ago
Can you give a test case that does not use spring?
But my guess is that DefaultTyping.NON_FINAL is the issue – records are final.
@yawkat
I write a test case without using Spring, same ObjectMapper configuration.
class MapperTest {
@Test
void test() throws JsonProcessingException {
final ObjectMapper mapper = new ObjectMapper()
.registerModule(new ParameterNamesModule(JsonCreator.Mode.DEFAULT))
.registerModule(new Jdk8Module())
.registerModule(new JavaTimeModule())
.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false)
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
mapper.activateDefaultTyping(mapper.getPolymorphicTypeValidator(),
DefaultTyping.NON_FINAL,
As.PROPERTY);
final RequestDto dto = new RequestDto(1L, "doljae", OffsetDateTime.now());
final RequestRecord record = new RequestRecord(1L, "doljae", OffsetDateTime.now());
final byte[] dtoBytes = mapper.writeValueAsBytes(dto);
final byte[] recordBytes = mapper.writeValueAsBytes(record);
// how to print the result?...
System.out.println(Arrays.toString(dtoBytes));
System.out.println(Arrays.toString(recordBytes));
}
}
@yawkat
Could you please explain in more detail about ParameterNamesModule
and activateDefaultTyping
?
The ObjectMapper
setting I am using is written with reference to the reference, so I do not fully understand the operation.
As you said, these settings seem to be the clues to the problem, and I googled and looked at the documentation, but I didn't quite understand.
Could you please explain in more detail? Or if you have a reference link that explains it in detail, please share.
i can reproduce your issue with the new test case, and yes it is because of DefaultTyping.NON_FINAL
. Changing it to DefaultTyping.EVERYTHING
resolves this. However it is somewhat problematic because it also adds default typing to unrelated places like the Long field.
A better alternative is to use mapper.writerFor(Object.class).writeValueAsBytes(record)
. This adds the type information even in the NON_FINAL case. However I can't say how to make spring use this approach.
@yawkat
Spring Data Redis uses ObjectMapper
in GenericJackson2JsonRedisSerializer.
RedisTemplate
just uses configured Serializer & Deserializer. Inside it, it use ObjectMapper
's serialization & deserialization methods.
Could you please explain in more detail about ParameterNamesModule and activateDefaultTyping? Plus, I am hoping to use it regardless of class or record under a common setting. In this situation, please suggest me how to get the ObjectMapper configuration.
@doljae getting the ObjectMapper configuration is something that Spring Data Redis users or maintainers can help with.
I think your problem can be divided in two parts:
But one other thing I would STRONGLY recommend: instead of attempting to serialize any given "Root value" directly, you really should instead use a wrapper if possible: something like:
public class Wrapper {
@JsonTypeInfo(....)
public Object wrapped; // or whatever name
// and/or getters, setters for accessing wrapped value
}
This will avoid many pitfalls, including problem of Java Type Erasure for the root value. It may also remove the need to active default typing completely (although this depends a little bit on kind of data you serialize).
In your case, for example, one problem is that Record
types are final
and non-polymorphic, and as such inclusion of type id may not occur as you'd expect, if such value is serialized directly.
@cowtowncoder , @yawkat
I improved my understanding by looking for settings of ObjectMapper
and writing test codes.
I still don't quite understand the principle. In other words, I did not fully understand exactly what the role of the ObjectMapper
settings used in this comment.
However, as @yawkat said, I understood that root class information was not saved when saving a Java Record
as a RedisTemplate
and then immediately taking it out as a Record. (Because Java Record is treated as a final class and due to the setting of ObjectMapper
, final class information is saved in redis without remaining when serialized.)
mapper.activateDefaultTyping(mapper.getPolymorphicTypeValidator(),
DefaultTyping.NON_FINAL,
As.PROPERTY); // <- this is the point
Here's what I can do in this situation:
DefaultTyping.EVERYTHING
Of course, this method is not recommended by @cowtowncoder. However, it is also the simplest way to solve it. And I'm not sure if it's good to use the Wrapper class just for this problem.
RedisTemplate
.I think this method is the most flexible. If you take it out as a String and deserialize it through ObjectMapper
, there is no problem because the target class is explicitly declared.
final String fromRedis = redisStringTemplate.opsForValue().get("key");
final RequestRecord deserialized = mapper.readValue(fromRedis, RequestRecord.class);
In this way, you do not need to modify ObjectMapper
related settings. I just can't use Java Record...
For reference, I chose method 3 in my current project (because I had to make a quick fix).
And on the Spring Data Redis side, from the latest version, it was changed to use DefaultTyping.EVERYTHING
as the ObjectMapper setting that is used by default in GenericJackson2JsonRedisSerializer
.
By the way, although I'm not a good English speaker, I couldn't find any really good material about detailed settings related to ObjectMapper and Jackson. (But there is a possibility that I did not understand it well)
If you have a reference that you can recommend, please share it. The more detailed and the more examples, the better.
Sorry, I do not think there is much good documentation regarding this particular configuration (default typing).
But one quick comment. This:
mapper.activateDefaultTyping(mapper.getPolymorphicTypeValidator(),
DefaultTyping.NON_FINAL,
As.PROPERTY); // <- this is the point
is something I strongly suggest you change, you should usually not use As.PROPERTY
with Default Typing.
Use As.WRAPPER_ARRAY
(or even As.WRAPPER_OBJECT
). Those will work with all types, are more efficient and simpler. As.PROPERTY
is ok with @JsonTypeInfo
, for types serialized as JSON Objects.
This may not be the main issue you are facing but I would avoid this setting anyway.
However, I do have one question: why do you not try the approach I suggested as the best solution:
public class Wrapper {
@JsonTypeInfo(include=JsonTypeInfo.As.WRAPPER_ARRAY, use = JsonTypeInfo.Id.CLASS)
public Object wrapped; // or whatever name
// and/or getters, setters for accessing wrapped value
}```
Then you would wrap value yourself on read/write: you would not need default typing; it would be guaranteed that Polymorphic Type Id was always written and so on.
This is what I would try myself.
@cowtowncoder Thank you for your answer. I answer your question.
AS.WRAPPER_OBJECT
. However, in many projects, it seems that AS.PROPERTY
is being used a lot.ObjectMapper
(limited to RedisTemplate
).Use As.WRAPPER_ARRAY (or even As.WRAPPER_OBJECT). Those will work with all types, are more efficient and simpler. As.PROPERTY is ok with @JsonTypeInfo, for types serialized as JSON Objects.
Thank you for your advice. I understood that using WRAPPER_OBJECT
is more universally available than using PROPERTY
.
I don't think this change will cause a side effects. Because the ObjectMapper
does serialization and de-serialization, so they will do serialization and de-serialization according to the setting. However, I think it needs to be checked because it was used as a common setting for the project.
However, the ObjectMapper
setting that I wrote in the comment seems to be the setting that I had been using before I participated in the project. Not only that, but it seems to be used in many places. Even the Spring Data Redis
project's GenericJackson2JsonRedisSerializer uses this setting as its default.
This setting that I applied to ObjectMapper
is not the global ObjectMapper
setting of the project, but the ObjectMapper
setting of Serializer, Deserializer used by RedisTemplate
of Spring Data Redis
. I'll post a PR on the Spring Data Redis
side about this.
However, I do have one question: why do you not try the approach I suggested as the best solution:
This is just because it is convenient not to use the Wrapper class. For example, suppose have a Target class and a Wrapper class, I can use it like the code snippet you suggested.
@Test
void test11() throws JsonProcessingException {
final RequestRecord targetClass = new RequestRecord(1L, "doljae", OffsetDateTime.now());
final RequestWrapper wrapperClass = new RequestWrapper(targetClass);
final ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new ParameterNamesModule(JsonCreator.Mode.DEFAULT))
.registerModule(new Jdk8Module())
.registerModule(new JavaTimeModule())
.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false)
.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
.configure(DeserializationFeature.ADJUST_DATES_TO_CONTEXT_TIME_ZONE, false);
final String serialized = mapper.writeValueAsString(wrapperClass);
System.out.println(serialized);
final RequestWrapper requestWrapper = mapper.readValue(serialized, RequestWrapper.class);
final RequestRecord wrapped = requestWrapper.getWrapped();
System.out.println(wrapped);
}
As in the sample code above, serialization and de-serialization using ObjectMapper
directly work well without any problems.
But the first direction I was going to do was to set and get objects directly to RedisTemplate
without any preprocessing. And to use this way, eventually, I need to provide information so that the ObjectMapper
can be deserialized when serializing to the Wrapper class as well. That is, the identifier information should be serialized together.
@DisplayName("record -> set() -> get() -> record -> X")
@Test
void test3() {
final RequestRecord targetClass = new RequestRecord(1L, "doljae", OffsetDateTime.now());
final RequestWrapper wrapperClass = new RequestWrapper(targetClass);
redisWrapperTemplate.opsForValue().set("key", wrapperClass);
// error!, cause RedisTemplate's deserializer does not know the wrapper class information
final RequestWrapper deserialized = redisWrapperTemplate.opsForValue().get("key");
final ObjectMapper mapper = new ObjectMapper();
.......
// works, cause we gives class information to ObjectMapper
final RequestWrapper requestWrapper = mapper.readValue(serialized, RequestWrapper.class);
.......
}
To get and set objects directly to RedisTemplate, information about the Wrapper class must also be provided when serializing. In other words, @JsonTypeInfo
should be used for the Wrapper class.
I think this part is a matter of style. If the type I use as a target class is an abstract class and I need to apply polymorphism, this approach would be fine. However, the method you suggested should only use the Wrapper class for DTO to deal with DTO, and even add additional annotations to DTO.
On the other hand, the way to take the common ObjectMapper
setting is to write code to the convention of our commonly used DTO class without separate Wrapper class and annotation.
Describe the bug Serialized result of java
Class
&Record
is different withRedisTemplate
.Version information 2.13.3
To Reproduce
Expected behavior Both class and record objects should be stored and deserialized normally in RedisTemplate.
Additional context
The issue occurred while using
Spring Data Redis
class, but sinceObjectMapper
does serialization and deserialization, I reported it to Jackson issue borad.You can reproduce this issue with this repository's test class (use redis docker image in
repo/docker/docker-compose.yml
)I found that if i adjust ObjectMapper Configuration, both Class & Record serialize & deserialize normally.