tarantool / queue

Create task queues, add and take jobs, monitor failed tasks
Other
237 stars 52 forks source link

Full scan in `queue.statistics()` #92

Open derElektrobesen opened 5 years ago

derElektrobesen commented 5 years ago
  1 local function build_stats(space)
  2     local stats = {tasks = {}, calls = {
  3         ack  = 0, bury  = 0, delete  = 0,
  4         kick = 0, put   = 0, release = 0,
  5         take = 0, touch = 0,
  6         -- for *ttl queues only
  7         ttl  = 0, ttr   = 0, delay   = 0,
  8     }}
  9
 10     local st = rawget(queue.stat, space) or {}
 11     local idx_tube = 1
 12
 13     -- add api calls stats
 14     for name, value in pairs(st) do
 15         if type(value) ~= 'function' and name ~= 'done' then
 16             stats['calls'][name] = value
 17         end
 18     end
 19
 20     -- add total tasks count
 21     stats['tasks']['total'] = box.space[space].index[idx_tube]:count()
 22
 23     -- add tasks by state count
 24     for i, s in pairs(state) do
 25         stats['tasks'][i:lower()] = box.space[space].index[idx_tube]:count(s)
 26     end
 27     stats['tasks']['done'] = st.done or 0
 28
 29     return stats
 30 end

This function calls count() 6 times on each queue.statistics(). This could be a reason of very aggressive CPU usage if there are very many tasks in the queue.

olegrok commented 4 years ago

In de0a346f17e690aed19384577ed5157476f8fa3a we removed redundant fullscan from build stats

We can think about implementing a view that build stats on start and increment it then (see https://github.com/tarantool/queue/pull/93)