TS.RANGE reports ERR TSDB: wrong fromTimestamp

[afarber]

Good evening, I have the following question please:

With Azure Redis Cache (Enterprise E10 with TimeSeries module enabled) it is possible to retrieve all timestamps from a time series for the key "12345678:5" by calling:

$ TS.RANGE 12345678:5 - +
 1) 1) (integer) 1693407731665
    2) 1
 2) 1) (integer) 1693407731694
    2) 1
 3) 1) (integer) 1693407731745
    2) 1
 4) 1) (integer) 1693407731771
    2) 1
....
18) 1) (integer) 1693407732075
    2) 1
19) 1) (integer) 1693407732103
    2) 1
20) 1) (integer) 1693407732127
    2) 1

But I am only interested in the timestamps/values of the recent 30 minutes (1800000 milliseconds).

So I calculate the timestamps "start" and "now" by the Lua script:

$ EVAL "local time = redis.call('TIME'); local now = time[1] * 1000 + time[2] / 1000; local start = now - tonumber(ARGV[1]); return { tonumber(ARGV[1]), start, now }" 1 12345678:5 1800000
1) (integer) 1800000
2) (integer) 1693493878001
3) (integer) 1693495678001

And then I try to call TS.RANGE using the Lua script:

$ EVAL "local time = redis.call('TIME'); local now = time[1] * 1000 + time[2] / 1000; local start = now - tonumber(ARGV[1]); local tsrange = redis.call('TS.RANGE', KEYS[1], start, now); return tsrange" 1 12345678:5 1800000
(error) ERR Error running script (call to f_a99152dc66083038fe6d6b9b6acef33495ef1960): @user_script:1: ERR TSDB: wrong fromTimestamp

Please advise me, what am I doing wrong?

My real background is that I try to call TS.ADD and TS.RANGE in my C# web app (the auto-expiration does not really seem to work, so I cannot use "TS.RANGE key - +", but have to specify the fromTimestamp, toTimestamp arguments to TS.RANGE explicitly):

private static readonly TimeSpan RETENTION_TIME = TimeSpan.FromMinutes(30);
// The script returns 0 if there are 20 or more timestamps in the past 30 min (1800000 ms).
// Otherwise it adds a new timestamp to the time series at the key and returns 1.
private const string LUA_SCRIPT =
@"
local sum = 0
local time = redis.call('TIME')
-- the final timestamp for TS.RANGE command in milliseconds
local now = time[1] * 1000 + time[2] / 1000
-- the start timestamp for TS.RANGE command in milliseconds
local start = now - tonumber(ARGV[1])
local tsrange = redis.call('TS.RANGE', KEYS[1], start, now)

for i = 1, #tsrange do
    -- calculate a sum of all values stored in the time series, from start to now
    sum = sum + tsrange[i][2]['ok']

    -- if there are enough timestamps in the time period, then just return 0
    if (sum >= 20) then
        return 0
    end
end

-- otherwise add the current timestamp and the value 1 to the time series
redis.call('TS.ADD', KEYS[1], '*', 1, 'RETENTION', ARGV[1], 'ON_DUPLICATE', 'SUM', 'ENCODING', 'UNCOMPRESSED')
return 1
";

private static async Task<bool> AddTimestampAsync(IDatabase db, string key, long retentionTime)
{
    RedisResult result = await db.ScriptEvaluateAsync(LUA_SCRIPT, new RedisKey[] { key }, new RedisValue[] { retentionTime });
    // return true if the Lua script returns 1, otherwise false
    return result.Type == ResultType.Integer && (int)result == 1;
}

I hope that I do not have to workaround my issue by modifying the for-loop above and comparing each timestamp one by one...

Thank you Alex

[afarber]

I would like to share a workaround here for anyone with the TS.RANGE "wrong fromTimestamp" or "wrong toTimestamp" problem - you have to get rid of the fractional part by using math.floor:

EVAL "local time = redis.call('TIME'); local now = math.floor(time[1] * 1000 + time[2] / 1000); local start = now - tonumber(ARGV[1]); local tsrange = redis.call('TS.RANGE', KEYS[1], start, now); return tsrange" 1 12345678:5 1800000
(empty array)

It seems like a bug in TimeSeries module, fractional timestamps shouldn't break the TS:RANGE command.

[LiorKogan]

As documented, timestamps are Unix time (integer, in milliseconds). The command parser expects a non-negative integer value and replies with an error if this is not the case.

[afarber]

Aren't all numbers in Lua floating point? Wouldn't it make sense to accept fractional timestamps too?

[LiorKogan]

Since RedisTimeSeries stores timestamps as 64-bit unsigned integers (milliseconds since the Unix epoch), accepting floating-point values would be misleading. We would need to round or truncate the which may be unexpected and could be considered erroneous by some users due to an alleged loss of resolution.

Note also that starting with Redis Stack 7.2 Triggers and functions should be preferred over LUA scripts.