[NEW] Redis Json, Redis Search module support?

[VagyokC4]

The problem/use-case that the feature addresses

I would like to use KeyDb, with the redis modules ReJson and RedisSearch

Description of the feature

Examples of, or an image showing KeyDb running Redis Json + Redis Search modules.

Alternatives you've considered

Redis Stack

Additional information

I am using KeyDb, but need to take advantage of the Json/Search modules in Redis, and so I have to downgrade from KeyDb back to Redis, unless you can provide a working image / instructions to get working redis modules with KeyDb.

I'd like to keep using KeyDb, and have these redis modules able to load, but if not, I will have to switch back to redis-stack.

[agosalvez]

Hi @VagyokC4 ,

Try to compile this image and use it.

I've done and work it fine.

Redis JSON Redis Search

Reach me or give me feedback if you need more help. Regards!

[VagyokC4]

Hi @VagyokC4 ,

Try to compile this image and use it.

I've done and work it fine.

Redis JSON Redis Search

Reach me or give me feedback if you need more help. Regards!

@agosalvez Are these using latest versions of Redis Json, and Redis Search?

UPDATE: Yeah, these are older versions, and I have issues that were resolved in newer versions, so I need to be able to use the latest versions of ReJson and Redis Search.

[agosalvez]

Hi @VagyokC4, in the Dockerfiles are fixed the versions in line 7.

The version of REDIS-JSON module is: v1.0.4 https://github.com/agosalvez/keydb-modules/blob/e31a93be9a793ea3d7faf67c6903a416f3672b88/redisJSON.dockerfile#L7

The version of REDIS-SEARCH module is: v1.6.13 https://github.com/agosalvez/keydb-modules/blob/e31a93be9a793ea3d7faf67c6903a416f3672b88/redisearch.dockerfile#L7

But you can modify this version to whatever you want.

UPDATE: I have an image built with reJSON module included. Feel free to use it to test if it works for you: https://hub.docker.com/layers/237880344/magicadry/keydb-redisjson/v1/images/sha256-141a328a847dee9a8cba38fc394c2cdd47c3597b7a670f4f186b565e5cad157a?context=repo

[VagyokC4]

The version of REDIS-JSON module is: v1.0.4 https://github.com/agosalvez/keydb-modules/blob/e31a93be9a793ea3d7faf67c6903a416f3672b88/redisJSON.dockerfile#L7

The version of REDIS-SEARCH module is: v1.6.13 https://github.com/agosalvez/keydb-modules/blob/e31a93be9a793ea3d7faf67c6903a416f3672b88/redisearch.dockerfile#L7

@agosalvez I had problems getting it to build on my side. Maybe you can try on your side with: REDIS-JSON v2.0.9 REDIS-SEARCH v2.4.9

Also, are you able to get them both in a single image?

[agosalvez]

@VagyokC4 I will try to build it with these versions and I will ansewer you.

[agosalvez]

@VagyokC4 try this image and give me feedback please.

https://hub.docker.com/layers/247330126/magicadry/keydb/v6.3.1-rejson-research/images/sha256-17ec046f18d807c902d97c077cef5f80cf4bdfed7f7c8ad574707346f85fa35f?context=repo

UPDATE: These versions seems no working fine, I can't load at start of keydb-server. Try it, I hope you'll get it!

[VagyokC4]

@VagyokC4 try this image and give me feedback please.

https://hub.docker.com/layers/247330126/magicadry/keydb/v6.3.1-rejson-research/images/sha256-17ec046f18d807c902d97c077cef5f80cf4bdfed7f7c8ad574707346f85fa35f?context=repo

UPDATE: These versions seems no working fine, I can't load at start of keydb-server. Try it, I hope you'll get it!

@agosalvez Yeah, I got the same problem too. Failed to start, and thus the reason for the issue.

[agosalvez]

Hi @VagyokC4 , I think the libraries are wrong. I'll try more and I'll tell you.

[agosalvez]

@VagyokC4 Have you try to compile with cargo the rejson and redisearch projects to get rejson.so and redisearch.so?

[VagyokC4]

@VagyokC4 Have you try to compile with cargo the rejson and redisearch projects to get rejson.so and redisearch.so?

@agosalvez I had no luck compiling. Only the older version.

[agosalvez]

@VagyokC4 There are a problem with GLIBC 2.29, because keydb image is older and has a libc6 Installed: 2.27-3ubuntu1.6

You have to rebuild keydb image or use another version of rejson :s I'm researching to solve this problem. Have you got a keydb image with rejson v2.X working fine?

[VagyokC4]

@VagyokC4 There are a problem with GLIBC 2.29, because keydb image is older and has a libc6 Installed: 2.27-3ubuntu1.6

You have to rebuild keydb image or use another version of rejson :s I'm researching to solve this problem. Have you got a keydb image with rejson v2.X working fine?

@agosalvez Sadly, No. That's why I opened the issue here. For KeyDB to be a drop in replacement, I was hoping the plugins too would be drop in, but it doesn't seem they are compatible. (Yet... )

Keep me posted on your progress.

[JohnSully]

I'm a bit confused with the issue here. Is it when you try to load the .so you get an error due to glibc versions being different? Can you provide an error message.

Due to the non-free licenses on the Redis modules I'm not able to provide supported versions but can help resolve issues you face as an end user trying to load them.

[agosalvez]

Hi @JohnSully, I use eqalpha/keydb:x86_64_v6.3.1 image adding librejson.so module inside of and run keydb-server with --loadmodule.

With rejson.so (version 1.0.4) works fine, but with librejson.so (2.0.9 version compiled with cargo build --release as mentioned in RedisJSON Github

This is my Dockerfile:

(My Dockerfile reference is: Dockerfile of pozetroninc)

ARG KEY_DB_VERSION=v6.3.1

ARG BUILD_BIN=/build/bin

FROM debian:10.3-slim AS builder

ARG BUILD_BIN ENV BUILD_BIN=${BUILD_BIN}

WORKDIR /build RUN mkdir -p ${BUILD_BIN}

ADD librejson.so ${BUILD_BIN} RUN chmod a+x ${BUILD_BIN}/librejson.so && \ ls -ltr ${BUILD_BIN}

FROM eqalpha/keydb:x8664${KEY_DB_VERSION}

ARG BUILD_BIN

ENV LIBDIR /usr/lib/redis/modules RUN mkdir -p ${LIBDIR}

COPY --from=builder ${BUILD_BIN}/* ${LIBDIR}/

CMD [ "keydb-server", \ "/etc/keydb/keydb.conf", \ "--loadmodule", "/usr/lib/redis/modules/librejson.so" ]

This is my log:

1:1:C 06 Jul 2022 13:48:49.823 # oO0OoO0OoO0Oo KeyDB is starting oO0OoO0OoO0Oo, 1:1:C 06 Jul 2022 13:48:49.823 # KeyDB version=6.3.1, bits=64, commit=ee16abf0, modified=1, pid=1, just started, 1:1:C 06 Jul 2022 13:48:49.823 # Configuration loaded, 1:1:M 06 Jul 2022 13:48:49.824 monotonic clock: POSIX clock_gettime, 1:1:M 06 Jul 2022 13:48:49.825 Running mode=standalone, port=6379., 1:1:M 06 Jul 2022 13:48:49.825 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128., 1:1:M 06 Jul 2022 13:48:49.825 # Server initialized, 1:1:M 06 Jul 2022 13:48:49.825 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect., 1:1:M 06 Jul 2022 13:48:49.830 # Module /usr/lib/redis/modules/librejson.so failed to load: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /usr/lib/redis/modules/librejson.so), 1:1:M 06 Jul 2022 13:48:49.830 # Can't load module from /usr/lib/redis/modules/librejson.so: server aborting

[JohnSully]

What base OS was the Redis module's image built on?

[agosalvez]

Sorry, I don't understand you. librejson.so it was created compiling the project, I got a .so file, without image docker.

The .so file It was generated in Ubuntu 20

[JohnSully]

The dockerfile is based on Ubuntu 18.04 LTS. You must compile your module on this release in order to be compatible with the KeyDB in the dockerfile.

[agosalvez]

Yesterday I compiled keydb with ubuntu:20 + rejson 2.0.9 with same results.

Now, I've just compiled a keydb image based on ubuntu:20 + rejson 2.0.11 module instead of 2.0.9 (in Ubuntu 20 too) and it works!

I don't know why rejson v2.0.9 not works, but now I've got the newest version of rejson.

Thanks @JohnSully

@VagyokC4 , there is my image: https://hub.docker.com/layers/keydb/magicadry/keydb/x86_64_v6.3.1_rejson_2.0.11_ubuntu_20/images/sha256-d440b8d0ebff992b8936b1086e6a4688f7b8be87d3e5b052e7a8a76e809ed225?context=repo

I hope it works for you.

Regards.

[JohnSully]

Thanks @agosalvez

Just remember modules should always be compiled on the same OS version as KeyDB (or Redis). Sometimes it will work if you don't but you are setting yourself up for weird bugs if you don't follow this advice. Unfortunately Linux distros don't take binary compatibility very seriously.

[VagyokC4]

@VagyokC4 , there is my image: https://hub.docker.com/layers/keydb/magicadry/keydb/x86_64_v6.3.1_rejson_2.0.11_ubuntu_20/images/sha256-d440b8d0ebff992b8936b1086e6a4688f7b8be87d3e5b052e7a8a76e809ed225?context=repo

I hope it works for you.

@agosalvez That did work! Are you able to load the other plugins as well that make up the Redis Stack as well, all into a single image? Maybe a KeyDb-Stack image?

These are the missing plugins:

RediSearch Queries, secondary indexing, and full-text search for Redis

RedisGraph A Graph database built on Redis

RedisTimeSeries Ingest and query time series data with Redis

RedisBloom Bloom filters and other probabilistic data structures for Redis

[JohnSully]

Closing as its not an active KeyDB bug. Feel free to continue the discussion.

[agosalvez]

@VagyokC4 ok!, I'll try but I don't know when I'll get KeyDB-Stack image. I'll do in my freetime.