spring-projects / spring-data-redis

Provides support to increase developer productivity in Java when using Redis, a key-value store. Uses familiar Spring concepts such as a template classes for core API usage and lightweight repository style data access.
https://spring.io/projects/spring-data-redis/
Apache License 2.0
1.77k stars 1.17k forks source link

missing _class causes failure when deserializing to ObjectRecord again since version 3.0.0 #2871

Open larskruse opened 8 months ago

larskruse commented 8 months ago

This was first mentioned by @lrozek in issue #2251 and implemented with issue #2198 in release 2.7.0-M3 . Hence, this issue strongly builds on their description of the problem.

However, it does not work from version 3.0.0 onwards. I could not find any mentioning of this being removed in the release notes. If it was removed deliberately, I apologize for the inconvenience.

The StreamMessageListenerContainerOptions#targetType provides information regarding what class should be used for deserialization (https://docs.spring.io/spring-data/redis/docs/current/api/org/springframework/data/redis/stream/StreamMessageListenerContainer.StreamMessageListenerContainerOptionsBuilder.html). Producing system should not be enforced to provide _class pl.lrozek.redis.stream.consumer.domain.TemperatureReadingDto field-value pair in a stream entry, as this is an implementation detail of a consuming system.

This is especially problematic if one steam is meant to be consumed by different services.

When _class field-value pair is missing following exception is thrown:

2024-03-14T18:57:42.088Z ERROR 1 --- [cTaskExecutor-1] ageListenerContainer$LoggingErrorHandler : Unexpected error occurred in scheduled task

org.springframework.core.convert.ConversionFailedException: Failed to convert from type [org.springframework.data.redis.connection.stream.StreamRecords$ByteMapBackedRecord] to type [pl.lrozek.redis.stream.consumer.domain.TemperatureReadingDto] for value [org.springframework.data.redis.connection.stream.StreamRecords$ByteMapBackedRecord@2b5ed41a]
 at org.springframework.data.redis.stream.StreamPollTask.convertRecord(StreamPollTask.java:178) ~[spring-data-redis-3.2.3.jar:3.2.3]
 at org.springframework.data.redis.stream.StreamPollTask.deserializeAndEmitRecords(StreamPollTask.java:156) ~[spring-data-redis-3.2.3.jar:3.2.3]
 at org.springframework.data.redis.stream.StreamPollTask.doLoop(StreamPollTask.java:128) ~[spring-data-redis-3.2.3.jar:3.2.3]
 at org.springframework.data.redis.stream.StreamPollTask.run(StreamPollTask.java:112) ~[spring-data-redis-3.2.3.jar:3.2.3]
 at java.base/java.lang.Thread.run(Thread.java:1583) ~[na:na]
Caused by: java.lang.IllegalArgumentException: Value must not be null
 at org.springframework.util.Assert.notNull(Assert.java:172) ~[spring-core-6.1.1.jar:6.1.1]
 at org.springframework.data.redis.connection.stream.Record.of(Record.java:99) ~[spring-data-redis-3.2.3.jar:3.2.3]
 at org.springframework.data.redis.connection.stream.MapRecord.toObjectRecord(MapRecord.java:139) ~[spring-data-redis-3.2.3.jar:3.2.3]
 at org.springframework.data.redis.core.StreamObjectMapper.toObjectRecord(StreamObjectMapper.java:137) ~[spring-data-redis-3.2.3.jar:3.2.3]
 at org.springframework.data.redis.core.StreamOperations.map(StreamOperations.java:623) ~[spring-data-redis-3.2.3.jar:3.2.3]
 at org.springframework.data.redis.stream.DefaultStreamMessageListenerContainer.lambda$getDeserializer$2(DefaultStreamMessageListenerContainer.java:217) ~[spring-data-redis-3.2.3.jar:3.2.3]
 at org.springframework.data.redis.stream.StreamPollTask.convertRecord(StreamPollTask.java:176) ~[spring-data-redis-3.2.3.jar:3.2.3]
 ... 4 common frames omitted

reproduction project and full description can be found at https://github.com/larskruse/redis-stream-deserialization-v3.0.0 which is an updated fork of @lrozek original project found at https://github.com/lrozek/redis-stream-deserialization.

I updated it to use spring boot version 3.2.0, java 21 and spring-data-redis version 3.2.3. However, it does not work starting with spring-data-redis version 3.0.0.

JasonWangBASF commented 5 months ago

I have the same issue..So sad. Please help to fix this asap.