hibiken / asynq

Simple, reliable, and efficient distributed task queue in Go
MIT License
9.65k stars 696 forks source link

[Question] Are there any way to run the tasks that's retention exceeds #923

Open arifmahmudrana opened 6 days ago

arifmahmudrana commented 6 days ago

Are there any way to run the tasks that's retention exceeds? e.g. when we call GetTaskInfo in the asynq.Inspector it returns the TaskInfo but when executing the task using the RunTask using the asynq.Inspector we get error. You are calling a redis script

var runTaskCmd = redis.NewScript(`
if redis.call("EXISTS", KEYS[1]) == 0 then
    return 0
end
local state, group = unpack(redis.call("HMGET", KEYS[1], "state", "group"))
if state == "active" then
    return -1
elseif state == "pending" then
    return -2
elseif state == "aggregating" then
    local n = redis.call("ZREM", ARGV[3] .. group, ARGV[1])
    if n == 0 then
        return redis.error_reply("internal error: task id not found in zset " .. tostring(ARGV[3] .. group))
    end
    if redis.call("ZCARD", ARGV[3] .. group) == 0 then
        redis.call("SREM", KEYS[3], group)
    end
else
    local n = redis.call("ZREM", ARGV[2] .. state, ARGV[1])
    if n == 0 then
        return redis.error_reply("internal error: task id not found in zset " .. tostring(ARGV[2] .. state))
    end
end
redis.call("LPUSH", KEYS[2], ARGV[1])
redis.call("HSET", KEYS[1], "state", "pending")
return 1
`)

So I assume it's erroring while calling local n = redis.call("ZREM", ARGV[2] .. state, ARGV[1]) because my task isn't in pending or active or in aggregating.

So are there anything I can do to add that to the queue may be by adding a record in the redis any idea?

kamikazechaser commented 5 days ago

because my task isn't in pending or active or in aggregating.

Yes, it might be completed. If you want to run a completed task, would copying the payload and running it again suffice?

arifmahmudrana commented 2 days ago

because my task isn't in pending or active or in aggregating.

Yes, it might be completed. If you want to run a completed task, would copying the payload and running it again suffice?

@kamikazechaser thanks for your suggestion yes I can clone the task that's one option but my question is whether it's possible without cloning e.g. by adding some record in some hashset etc. I would prefer directly adding records in the redis without going one by one in the task list

kamikazechaser commented 1 day ago

Well technically you could control the states within the hashset, but this is not documented. Maybe have a look at Inspector.RunAllArchivedTasks and see if you can modify it for your use case.