sewenew / redis-plus-plus

Redis client written in C++
Apache License 2.0
1.64k stars 351 forks source link

xadd without auto generated id #398

Closed bigsamich closed 2 years ago

bigsamich commented 2 years ago

Id like to do the following; however, it does not seem to be working, not sure if i am missing something. I am generating time stamps myself and want to use that as the time-id

string timeID(to_string(timestamp.secondsGet())+"-"+ to_string(timestamp.nanoSecondsGet())); auto id = redis.xadd("key", timeID, attrs.begin(), attrs.end());

sewenew commented 2 years ago

The code looks good to me. What's exactly the problem?

Regards

bigsamich commented 2 years ago

The xadd throws an exception.

I have tried other time ID, but can only get "*" to work. Could this be on the redis side rejecting the xadd?

The time-ids are timestamps , so should be incrementing .

wingunder commented 2 years ago

Hi @bigsamich,

Please use try-catch to see what the problem is. It could be that you're trying to insert an id that's smaller than the last inserted id, in which case the Redis server will refuse to add the entry. I demonstrated this behaviour in the example below:

#include <iostream>
#include <sw/redis++/redis++.h>

template <typename T>
void xadd(T& redisInstance, const std::string& key, const std::string& timeId, const std::vector<std::pair<std::string, std::string>>& attrs)
{
        try {
                auto id = redisInstance.xadd(key, timeId, attrs.begin(), attrs.end());
                std::cout << "inserted id: " << id << std::endl;
        } catch (const std::exception &err) {
                std::cerr << "xadd(" << key << ", " << timeId << ", ...) failed: " << err.what() << std::endl;
        }
}

int main()
{
        sw::redis::ConnectionOptions connOpts;
        connOpts.host = "127.0.0.1";
        connOpts.port = 6379;
        sw::redis::Redis redisInstance(connOpts);

        const std::string key = "test_stream";

        std::vector<std::pair<std::string, std::string>> attrs = {{"f1", "v1"}, {"f2", "v2"}};
        xadd(redisInstance, key, "1-0", attrs);
        xadd(redisInstance, key, "1-1", attrs);
        xadd(redisInstance, key, "2-1", attrs);
        xadd(redisInstance, key, "2-0", attrs);

        try {
                redisInstance.del(key);
        } catch (const std::exception &err) {
                std::cerr << "del err: " << err.what() << std::endl;
        }
        return 0;
}

Output:

inserted id: 1-0
inserted id: 1-1
inserted id: 2-1
xadd(test_stream, 2-0, ...) failed: ERR The ID specified in XADD is equal or smaller than the target stream top item

Also please look at the redis-plus-plus unit tests for streams: https://github.com/sewenew/redis-plus-plus/blob/master/test/src/sw/redis%2B%2B/stream_cmds_test.hpp

I hope this helped. Regards

sewenew commented 2 years ago

The time-ids are timestamps , so should be incrementing .

NO. The system clock is NOT monotonic (that's why we have std::chrono::system_clock and std::chrono::steady_clock), especially when using nanoseconds. As @wingunder mentioned, you can print out the error messages and your time-ids to debug it.

Regards

bigsamich commented 2 years ago

Hi All, Im closing this thread. I figured out my problem with your suggestions. I was switching back and forth between timeID and "" My time ID is in the form seconds-nanoseconds where as "" is in microseconds-id. So redis was throwing as expected with XADD is equal or smaller than the target stream top item stream.