search

dmwm / CRABServer

15 stars 38 forks source link

separate CRAB/Condor I/O on schedds #5937

Open belforte opened 5 years ago

belforte commented 5 years ago

THis is not code, but let's use GIT to track it.

we should add one small volume to schedd and use it only for condor queue and history. So that schedd does not suffer when some crab taks gets out of hand and makes too much I/O. See thread with Brian.


On 23/07/2019 16:03, Bockelman, Brian wrote:
> For the schedd, I would recommend moving $(LOG) to somewhere on the system disk and
setting HISTORY and JOB_QUEUE_LOG to somewhere else besides the spool (default is
$(SPOOL)/job_queue.log).  This should be done by hand as you want to carefully copy over all
the contents to the new location.  I would suggest the procedure go through a reasonable
change process since there's room for human screw-ups (i.e., don't do this late at night or
without testing).
> 
> Finally, if we set:
> 
> PREEN=
> 
> (that is, unset the PREEN variable for the CRAB schedds) we should prevent a lock-up of the
schedd once every 24 hours.  That won't solve all our problems but certainly won't hurt.
> 
> Brian
belforte commented 3 years ago

We are out of crisis mode since we have multithreaded negotiator. So priority is reduced. But it would be good to do this before next spring.

belforte commented 3 years ago

There is no clear indication that we need this. Putting on Hold