How are Buffers converted to string internally?

[ljubadr]

I apologize in advance for asking the question here - I've originally posted my question on stackoverflow, but after 10 days I still didn't manage to get the answer that I was looking for... I thought I might try my luck here :smile:. If it's not ok to ask the question here, let me know and I'll remove it...

I'm not having any issues with the library, but I'm trying to figure out how Buffer is being handled in node-redis library. I have my nodejs script that fetches some data from database, serializes and gzips the result and then stores that result in redis. Later I read this value in php script, so I need to match what php script expects

In node gzip library returns Buffer and what I'm wondering is how does node-redis convert this Buffer to string internally?

When I pass Buffer (from gzip) to redis, it will be saved in the format that my php script can read and decode properly

await client.setex('compressed', 3600, compressed);

Value in redis (result is from redis desktop manager terminal)

redis1:0>get "compressed'
"\x1F\x8B\x08\x00\x00\x00\x00\x00\x02\x03\xABVJI,IT\xB2R*I-.Q\xAA\x05\x00\xC0\xE3/\xAC\x0F\x00\x00\x00"

But when I pass Buffer.toString() to redis, php won't be able to read and decode

await client.setex('compressed', 3600, compressed.toString());

Value in redis

redis1:0>get "compressed-toString"
"\x1F\xEF\xBF\xBD\x08\x00\x00\x00\x00\x00\x02\x03\xEF\xBF\xBDVJI,IT\xEF\xBF\xBDR*I-.Q\xEF\xBF\xBD\x05\x00\xEF\xBF\xBD\xEF\xBF\xBD/\xEF\xBF\xBD\x0F\x00\x00\x00"

Now I'm trying to figure out how exactly is node-redis converting this buffer to string internally? Why are these 2 values different? Best that I could find is this stream write, but I'm still not clear on how exactly it works and why are the redis values different...

Any help is appreciated!

---

Test example

I've created simple example

---

Environment

• Node.js Version: v12.18.3

• Redis Version: 5.0.5 (redis:5.0.5-alpine)

• Platform: Ubuntu 20.04

[SimonAbdullah]

I'm having a similar issue. When I pass a Buffer argument as message to the publish method, things go as expected. However, an unexpected behavior happens when I pass a Buffer argument to eval method.

The same Buffer value is serialized differently: Using publish I get: \x00\xd7ME\xc2\x7f\x9a\xc1\xc9I\x16\x12uX\x11\xc6 Using eval I get: \x00\xef\xbf\xbdME\xef\xbf\xbd\x7f\xef\xbf\xbd\xef\xbf\xbd\xef\xbf\xbdI\x16\x12uX\x11\xef\xbf\xbd

The publish method supports both strings and Buffers, but eval only supports strings. However, I can't serialize the buffer myself.