cpp-redis / cpp_redis

C++11 Lightweight Redis client: async, thread-safe, no dependency, pipelining, multi-platform
MIT License
713 stars 198 forks source link

Binary data broken? #74

Open jgwinner opened 3 years ago

jgwinner commented 3 years ago

Is your feature request related to a problem? Please describe. I can't store binary data in Redis via cpp_redis, although I can with other languages

Describe the solution you'd like Is there a way to call "set" with something other than a std::string / i.e. a raw buffer and size?

Describe alternatives you've considered I'm temporarily (for about 40 seconds) storing some PNG files into Redis. Unfortunately, the .set command uses std::string. Normally I would applaud this, but in this case it's causing a double allocation and a corrupt file.

ALL of the std::string initializers either walk the string and set every character to zero, or do a deep copy and null terimate the copy. They also null terminate the string. A binary file, which IS compatible with Redis, doesn't have to be null terminated. If I copy a PNG file with a null at the end, it may cause the file to be corrupt.

I'm dealing with a few hundred K, with the largest about a meg and a half, so creating a C++ string by allocating bytes that will subsequently be immedietely overwritten by OpenGL buffer copies seems wasteful.

So essentially, cpp_redis cannot store data that Redis is capable of storing? Or is there a way around this?

I looked at std::string_view but I can't quite coerce that into std:string. I'm using QT library, which does have a way to initialize a raw buffer to a QString, but I found out that .toStdString will also do a deep copy to put the null terminated string.

Regarding other languages, for example, with php/Predis:

$client = new Predis\Client();
$client->set('profile_picture', file_get_contents("profile.png"));//value is binary

Additional context

I'm using OpenGL to create some small thumbnails for a graphical editor. Right now, it can take up to 30 seconds at hosting to open a webcache and dump the data there, and we're using the premium service. It's mostly as there are dozens of small files, and each "file/open" takes a long time. We were attempting to create a Redis Cashe for this data until the data is persisted to disk. We are would lose clients dramatically if every change took 30 seconds until it was visible.

jgwinner commented 3 years ago

Anyone? :(

jgwinner commented 3 years ago

It was a critical feature for us. Taking a look at a QT library that does Redis.

larkwiot commented 1 year ago

I think you can create a std::string from bytes data?