moving reaper to job_master

[GoogleCodeExporter]

Bug opened based on this discussion:

dormando
though it's almost an interesting idea to retire the reaper proc and use 
job_master
by almost I mean now I've convinced myself that this is a good idea
marking stuff as dead should queue more intelligently
or at least give you the ability to queue it
lilnick
that would be nice - because I've considered building a whole system to manage 
the queue of bad disks, mark them dead as the replicate queue allows, etc
dormando
well I don't see the thing I described as being more than ten lines of perl and 
"solving" the issue as well
we still have no good way of automatically detecting if a drive is gone for 
good or not, so a human still needs to hit some button somewhere
(and the sooner the better)
lilnick
yeah, but our monitoring system can mostly do that
dormando
lilnick: want to open a bug report about moving reaper to job_master? that 
might not actually be too hard
dormando
I just have the guts all on the floor at the moment redoing some internals
lilnick
I'll open a bug w/ this conversation in it - not clear exactly on what moving 
reaper to job_master means but it'll be in the subject
dormando
that's all I need, thanks :)

Original issue reported on code.google.com by adnich...@gmail.com on 22 Apr 2011 at 8:28

[GoogleCodeExporter]

This seems like less of a good idea. Too much work being done inside the 
JobMaster.

However, it seems like a fine idea to add a DB setting which tells the reaper 
to back off if the file_to_replicate queue is too large.

Ultimately, it may be best to have JobMaster fetch the fid list, then queue out 
the fids to remove to local reaper workers. Presently, adding multiple reaper 
workers makes them step on each other.

We'll see which one I'm more up to hacking in when the next release comes up.

Original comment by dorma...@rydia.net on 29 Jul 2011 at 6:48

• Changed state: Accepted

[GoogleCodeExporter]

this actually got fixed already.

queue_size_for_reaper/queue_rate_for_reaper

just forgot to close it out.

Original comment by dorma...@rydia.net on 18 Jan 2013 at 9:36

• Changed state: Fixed