search

spring-projects / spring-data-redis

Provides support to increase developer productivity in Java when using Redis, a key-value store. Uses familiar Spring concepts such as a template classes for core API usage and lightweight repository style data access.
https://spring.io/projects/spring-data-redis/
Apache License 2.0
1.78k stars 1.17k forks source link

Add support for external/custom marshalling/unmarshaling implementations #2547

Open uglide opened 1 year ago

uglide commented 1 year ago

Problems with the current state:

Currently, Spring Data Redis allows marshalling (serializing) POJOs using Redis Hashes with @RedisHash annotation. This approach doesn’t allow efficiently store and querying nested objects like discussed here. Also, the selection of the proper serializer for POJOs is confusing for the developers.

Proposed solution:

Introduce a new API that allows extending Spring Data Redis with external/custom marshalling/unmarshaling implementations.

For example, it will allow us to build JSON-based serialization to store POJOs as JSON documents in Redis. JSON.GET allows to retrieve and update embedded data efficiently with JSONPath, which can be used to significantly improve the performance of retrieving nested objects.

An example of a JSON-based serialization can be found here https://github.com/redis/redis-om-spring/blob/main/demos/roms-documents/src/main/java/com/redis/om/documents/domain/Company.java#L27

This can be combined with https://github.com/spring-projects/spring-data-redis/issues/2546 to allow indexing JSON documents in addition to Hashes.

mp911de commented 1 year ago

We appreciate your suggestion. Our scope is to support Redis core functionality to cover what an out-of-the-box Redis provides.

RedisJSON is an add-on and not part of Redis Core. We deliberately do not provide support for functionality shipped via modules as we cannot reason what features are available on a server installation. If JSON support were part of the core, we would consider adding support.

With modules, the ecosystem is open-ended. So, we cannot provide integrations as the next module author would like to see support for their module as well.

uglide commented 1 year ago

@mp911de JSON in Redis is becoming an industry standard available in Enterprise offerings like Redis Enterprise, AWS ElastiCache and MemoryDB. Considering the fact that Spring Data Redis is an essential part of the ecosystem highly adopted by enterprises all over the world, it's important to make SDR more extensible to give more power to developers and allow them to handle more use-cases with SDR instead of reinventing the wheel as in this case.

I've made some changes to the issue description to clarify our intentions better, and I hope you will reconsider your decision based on this new information.

mp911de commented 1 year ago

Thanks for the additional detail. With JSON in Redis becoming more popular, the natural consequence should be moving Redis JSON support into the core. Once this is done, we're happy to reconsider adding this feature to Spring Data Redis.

mp911de commented 1 year ago

I reopened this ticket open to investigate whether we can adjust the feature scope and make a subset of this happen.

uglide commented 1 year ago

@mp911de May I ask if there is a specific reason why only features available under a particular license are being supported? I'm having a difficult time understanding the difference between the licensing of Redis Stack and MongoDB available under the same SSPL, and I believe that Spring Data Redis and Spring Data Mongo should support all the latest features of their corresponding products.

To be clear, we are not demanding any new feature development. We simply wanted to bring to your attention the frustration that many developers are experiencing with the current limitations. Our goal is to collaborate with you and other members of the Spring team to find a solution that will provide a more powerful and easy-to-use API. We have already completed most of the necessary code in https://github.com/redis/redis-om-spring under the MIT license, and we would be happy to backport it and submit a pull request to SDR once we agree on the API design and add extensibility points in the SDR codebase.

mp911de commented 1 year ago

Sure. Since their inception, all supported modules have integrated with open-source technologies. MongoDB has been OSS (according to OSI) until 2018. With that, Spring Data MongoDB was 8 years+ available to our community before MongoDB switched to SSPL. We did not want to stop just there, leaving our users without an alternative.

However, we do not provide integration with commercial modules such as their Cloud offering (e.g. Atlas Search).

Over time, database vendors keep adding more functionality to their products than we could ever catch up with our limited team. Spring Data MongoDB, Redis, JPA, JDBC, and a few more modules are maintained by a small team without being supported with maintenance or significant other contributions from database vendors. In the data space, we receive almost exclusively contributions from independent contributors that are not affiliated with data store vendors.

With all that change in the technology sector, where vendors heavily leverage open-source software, we believe that if a vendor wants to optimize a piece of software to maximize their impact, then the only fair approach is getting heavily involved into open-source maintenance, contribution, and becoming accountable to a large extent.

Especially in the Redis space we understand that there is no single way to meet the needs of Redis users and that flexibility is key. That is, why we reopened the ticket as we're going to take your request to our team for discussion.