search

redis / jedis

Redis Java client
MIT License
11.88k stars 3.87k forks source link

StreamEntry (XREAD) does not support binary because it uses Map<String, String> #3566

Open raylau1234 opened 1 year ago

raylau1234 commented 1 year ago

StreamEntry uses a Map<String, String> for fields.

As a result, binary data which contains illegal utf-8 sequences get mangled.

Reproduce by using XADD with a key or value which contains an illegal utf-8 sequence like 0xc3 0x28.

Retrieve via XREAD.

Expected result -- API to get byte array instead of a String, which is lossy with illegal utf-8 sequences.

sazzad16 commented 1 year ago

First of all, what is XGET?

Secondly, there are xrange(byte[] key, ...) methods. Don't these help?

raylau1234 commented 1 year ago

Sorry, I meant XREAD.

You are correct on the alternate XRANGE signature. But I am really using XREAD. Let me update the title and to clarify.... XREAD only is problematic.

raylau1234 commented 1 year ago

I see there are alternate signatures for xread as well. Let me see of those help. I found the StreamEntry ones first. Give me a few minutes to explore and will udpate.

raylau1234 commented 1 year ago

OK, List<byte[]> xread(XReadParams, Entry<byte[], byte[]>... streams) is a possible workaround, but is cumbersome bec I need to deserialize the byte into the map of key / message id / values. i.e. I need to replicate everything that List<Map, Entry<String, List<StreamEntry>>> xread(XReadParams, Map<String, StreamEntryID>) is doing in terms of decoding with STREAM_READ_RESPONSE, STREAM_ENTRY_LIST, etc. (but skipping the SafeEncoder.encode() and keeping the binary there).