Tasks in 'taken' state

[fsitedev]

We have a number of random tasks in tarantool queue having odd state 'taken'. At the same time we have no opened sockets(sessions) associated with such tasks. To check we have phisically rebooted all applications and proxies connected to tarantool queue and found no changes in theirs states. 'ttl' option is the only thing that saves us from accumulating such broken tasks for today.

We have noticed some warnings in application:

/usr/share/tarantool/queue/abstract.lua:72: Task was not taken in the session

but these warnings have no strict correlation with broken tasks appearance, moreover this warning doesn't report a task number to investigate on.

Here is an example of such task from space 'box.space.common':

[17788457, 't', 1482752293369308, 86400000000, 15768000000000000, 9, 1482665893369308,
  {'handler': 'minute_log_prepare', 'volume': 500, 'value': 1482665760}]

Session is not found for this task:

box.space._queue_taken:pairs():grep(function (x) return x[3] == 17788457 end):length()
--- 0

Our tube init:

queue.create_tube('common', 'fifottl', {on_task_change = on_task_change_cb, if_not_exists = true})

Tarantool 1.6.8-771-g85df490

[bigbes]

Any update? Can you reproduce it? Maybe you're trying to modify task in on_task_change_cb?

[bigbes]

Problem was in on_task_change callback trying to modify task (delete them and e.t.c.).